Anyone know what could cause the microstuttering in Keen 4-6?

Here, you can get help with anything or just chat about the original Commander Keen games.
clb
Grunt
Posts: 13
Joined: Mon Sep 04, 2023 21:34

Anyone know what could cause the microstuttering in Keen 4-6?

Post by clb »

Hey all,

Keen 4-6 are common to stutter in scrolling. There are several different reasons and causes; one of those is due to vertical refresh synchronization code on certain hardware that was extensively discussed in viewtopic.php?t=4084 .

Recently, I have been researching the quirky behavior of different VGA cards and why Keen would stutter specifically on only some of them.

I did get into some insights, more about those in this thread: https://www.vogons.org/viewtopic.php?f=63&t=96028

But if we ignore hardware quirks for the moment, and restrict ourselves to running Keen only on "best possible" or "most compatible" VGA card/PC setup that does not have any of the undesirable hardware behavior, and even if we run any of the community patches that improve the vertical refresh in Keen games, even then, I feel that the scrolling issue has not yet been adequately addressed: in my tests I find that Keen still exhibits this kind of continuous "microstuttering" that I cannot find the source of.

The game intends to draw a new frame every second vblank to obtain 35fps on 70hz VGA. But no matter what VGA card I use on my 80 MHz 486 PC and on vanilla MS-DOS 6.22 without any TSRs, or which version of Keen I try (different versions of Keen 4 1.0-1.4, Keen 5 1.0, 1.4), with King Duke's patches or without, I find that Keen manages to mostly (>99% frames) render in that smooth 35 fps - but every 5-10 seconds, it misses a single frame, which produces this kind of microstuttering effect. There does not seem to be any type of regular periodicity to these frame misses.

Also I find that enabling or disabling the PC turbo button (I think my Turbo button drops the PC to around 25 MHz from 80 MHz) causes no deterioration or improvement to the frequency of these microstutters.

This effect is best seen when Keen is climbing a pole upwards, where I think the game intends to scroll the screen at the slowest possible smooth rate, a constant one pixel every 2 frames (35 pixels/second) pace.

I thought that this issue could have been due to the game vsync code maybe still being subtly buggy, so I broke out Ida disassembler, searched through locations in Keen5 1.4 code that perform accesses to I/O port 3DAh and 3D4h, and attempted to patch those using CK5PATCH with custom assembly code to remove the synchronization altogether. (I found three such locations in that version)

Although I feel that I may have failed to do so properly, since I find I am unable to make the game run unboundedly fast even, so not completely sure what to make of my experiment.

Does anyone know the type of microstuttering that I am talking about? I.e. the game is smooth for the most part, but not 100% butter smooth.

Comparing to my testbench for different hardware panning synchronization behavior at https://github.com/juj/crt_terminator/b ... P#L66-L172 , most of the methods there are able to produce a 100% smooth update with a very deterministic idle time between frames, so I have a good impression at least that the PC would not be stalling.

Do the Keen games just contain that much "lopsided" or "fluctuating" computation that they inevitably need to do too much computation during some frames that will throw them off to miss a frame? (I would find that odd if that's the case since adjusting CPU between 25 MHz -> 80 MHz seemingly has no effect on the frequency) If so, is there any hint on what those heavy computations would be? They don't seem to have a deterministically repeating pattern to them at least. And the microstuttering also happens in the main intro with the big letters scrolling from left to right.

Could it be related to audio processing? Maybe not, since if I disable all audio, the microstuttering is still there, and it does not seem to become more infrequent by removing audio.

Is it just my PC after all? Should I grow my vintage PC catalog with an overkill faster > 100 MHz (> 1 GHz?) box to ensure Keen will never miss a frame?

(btw on the performance front, I find there is some kind of missed performance situation in Keen that it wants to redraw also secret candies that will normally never show up to the player; just to hide them afterwards by painting walls over them. Secret candies could have been completely ignored in the paint routines - though that's unrelated to this microstuttering issue)

I don't have any joysticks connected, just playing with the keyboard.

Any thoughts or clues? :garg
User avatar
Multimania
Vortininja
Posts: 77
Joined: Sat Nov 10, 2007 8:10
Location: Hiding in a small, cramped corner of the BwB megarocket.
Contact:

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by Multimania »

I'm not as much an expert on this as K1n9_Duk3, but:

tl;dr: Keen doesn't use the EGA/VGA for timing, it uses the PIT. While it does try to update the screen during vblank to avoid tearing, this is not fed back into the game timing, so they can go out of sync.

Timing in Keen 4–6 is quite complicated, and can be split into two parts:
- Game timing (based on the PIT, not the VGA), which operates on a ~70.068Hz clock. In-game, each frame takes between 2 and 5 clock ticks (limiting the framerate to 35Hz, and causing the game physics to slow down if it drops below ~14Hz). When playing back a demo, this is fixed at 3 ticks (~23Hz).
- Sync to vblank. The game tries to sync each frame to the vblank, but the exact method used differs with version and settings. The actual game clock above is not affected by vblank (except if it causes a frame to be missed).

There are several different sync-to-vblank approaches, depending on the version, for example:
- Keen Dreams: Wait for hblank, set CRTC start, wait for vblank start, set PEL PANNING.
- v1.4: Wait for a sufficiently long period of display disable, without the vsync pulse active, then update both CRTC start and PEL panning. If it takes more than 3 ticks (with the 70Hz PIT timer), give up and tear. The length of non-vsync DD waited for is controlled by the "Fix Jerky Motion" argument.

Other versions (the Keen 4 demo version, K1n9_Duk3's patches, etc) have their own slightly different rules.

Another possible cause of this kind of stutter is the "SVGA Compatibility" option, which disables scanout crossing the "wrap-around" point. If you scroll too far, the game has to copy the screen elsewhere in video memory, which is expensive enough to cause a hitch.

As for the secret items being drawn and then overwritten, that's just a side effect of how sprites and foreground tiles are handled generally: all sprites are redrawn when they update (move or change animation frame), then any foreground tile they touch is redrawn. This handles the case where a sprite is only partly behind a foreground tile, which the game otherwise doesn't track.
clb
Grunt
Posts: 13
Joined: Mon Sep 04, 2023 21:34

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by clb »

Thanks for the informative reply.

I had examined the disassembly of the PIT0 interrupt function, but had glanced over it since I saw it was set to run several hundred times a second, and speed depended on if Adlib or PC Speaker was set, so thought it was for audio only.

But now I realize that it is running at either 8*70 or 2*70 Hz, and it does increment a counter at 1:8 or 1:2 rate, which ends up being the global frame counter. And indeed there is a busy loop that spins to wait for two elapses of that counter. That will definitely cause the microstuttering.

I'll see if I can poke around this area. Thanks for the pointers!
clb
Grunt
Posts: 13
Joined: Mon Sep 04, 2023 21:34

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by clb »

I've got a first try idea put together of how to examine what the resulting effect would be if I synchronized the timings between PIT0 and the VGA vertical refresh rate. What I'd like to do is to change the line

TimeCount++;

at

https://github.com/CatacombGames/Cataco ... _SD.C#L745

into the assembly instructions

mov dx, 126h
in al, dx
mov ah, al
sub ah, [prevFrameCounter]
mov [prevFrameCounter], al
add [TimeCount], ah

where prevFrameCounter would be a new byte-sized global variable.

Though I am not quite sure how to achieve that with CKPATCH. I don't know how to locate the patch offset, and then I have an issue that my added code is longer than the code it replaces, so something creative would need to be done there. And I'd need to find an address for a global variable somewhere.

I tried the approach of trying to compile the recreated Keen sources at viewtopic.php?p=108822#p108822 so I would be able to avoid needing to patch on the fly, but I ran into an odd problem there - not sure how to proceed with that approach.
clb
Grunt
Posts: 13
Joined: Mon Sep 04, 2023 21:34

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by clb »

Thanks Multimania for the help in the thread viewtopic.php?t=11505&start=45

I was able to change the synchronization code to remove reliance on PIT0 game timing, and replacing it with pure vsync timing with some CRT Terminator's assistance ( https://oummg.com/ ), I am able to get a perfectly smooth 35Hz game update with no frame drops in a few minute long test climbing up and down a pole. Although that seems to only work on 80 MHz turbo state. At ~25MHz the PC is not fast enough.

I'll give this some more test and polish and see if this could be generalized to not need CRT Terminator's help.
User avatar
Multimania
Vortininja
Posts: 77
Joined: Sat Nov 10, 2007 8:10
Location: Hiding in a small, cramped corner of the BwB megarocket.
Contact:

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by Multimania »

Glad it's working!

I assume that the port 126h read is a frame counter from the CRT Terminator, then?

Personally, I wondered whether you could do something with the maligned VGA vblank IRQ (which should let you count vblanks easily). Except it doesn't work on most cards.

Whatever you do, you'll probably still need a fallback to the PIT timer, as some systems (original EGA, a bunch of laptops with LCDs, some capture setups using the vga240 tsr, etc) use a 60Hz mode for mode 0Dh, so Keen will run too slowly if synced to vblank. (That might be preferred, but is different enough it probably shouldn't be the default.)
clb
Grunt
Posts: 13
Joined: Mon Sep 04, 2023 21:34

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by clb »

Multimania wrote: Wed Sep 06, 2023 14:57 Glad it's working!

I assume that the port 126h read is a frame counter from the CRT Terminator, then?
That's right, 126h is a frame counter. However, it is a little bit special one at that. (below)
Multimania wrote: Wed Sep 06, 2023 14:57
Personally, I wondered whether you could do something with the maligned VGA vblank IRQ (which should let you count vblanks easily). Except it doesn't work on most cards.
I was experimenting with the vertical retrace IRQ recently, at https://github.com/juj/crt_terminator/b ... /VINTR.CPP , and that would be a nice way to do timing, and should work better for incrementing the game time counter than PIT0. Where supported, it would definitely provide a smoother game time counter.

However, unfortunately IBM botched their implementation of VGA hardware scrolling and vertical refresh synchronization. A couple of hardware quirks/bugs/misdesigns combine to make it really difficult to work on a "stock" system.

It turns out that they latch the VGA scroll register at vertical retrace start, exactly when the IRQ fires (and the vertical retrace status register at 3DAh bit 3 asserts). So when one receives the vertical retrace signal (either by polling or interrupts), it is already "too late" to scroll the screen.

Well, players might not notice a one frame delay if that was the only issue, but then IBM doubly botched it by having the Pixel Shift Count register latch at a different time: at the end of vertical retrace, creating a really annoying synchronization problem. This type of latching problem is the root cause of the ATI cards having the tearing scrolling in Keen in the first place (ATI latches Pixel Shift at each hblank). I've analyzed parts of this issue at https://www.vogons.org/viewtopic.php?f=63&t=96028 in more detail.

Because of this scroll register latching synchronization "bug"/misdesign in the hardware, it becomes imperative to be able to synchronize precisely to the start of vertical blank, which is the only compatible position where the scroll registers can be updated so it works on all hardware. But doing this synchronization is extremely awkward, and is what led to Keen having that "let's search for arbitrary 5 consecutive I/Os of display inactive" (later doubled to 10 I/Os with the Fix Jerky Motion option) mechanism.

Alternative sync mechanism might attempt to double sync - I think I saw some Keen version doing that, where Display Address register was programmed in active display time, and Pixel Shift was programmed at start of vblank, but that is difficult to do without disabling interrupts.

A big issue exists with interrupts and polling to wait for vblank, or vsync: one cannot disable interrupts during this wait, since that would kill sound interrupts, but then that risks missing the actual vblank/vsync.

So what CRT Terminator does it provides better registers that enable one to implement a more synchronized wait for vblank start, without needing to disable interrupts for a long period of time. Port 126h is a frame counter register, that advances right after the last visible pixel of a frame, i.e. at the start of hblank of the scanline that will lead to vblank. Then port 127h is a "scanlines until vblank" wait register, that counts how many scanlines there are left until the start of vblank.

This way code waiting for a vblank can do so for the most part without needing to disable interrupts, and only when there are only few scanlines left until the important timing sensitive moment, then interrupts can be enabled to hit the right moment precisely. The code for this is a bit more complex than what I showed in the earlier comment, and it looks like this: https://github.com/juj/crt_terminator/b ... #L289-L331

With that, I find I get smooth 35fps.

Well, in fact, I find that I can upgrade the game to get smooth 70fps by adjusting the game to advance one tic at a time in https://github.com/CatacombGames/Cataco ... 1462-L1467 (essentially setting MINTICS=1). No idea yet if that might have a chance of breaking any game logic, but yay, I'm happy!
Multimania wrote: Wed Sep 06, 2023 14:57
Whatever you do, you'll probably still need a fallback to the PIT timer, as some systems (original EGA, a bunch of laptops with LCDs, some capture setups using the vga240 tsr, etc) use a 60Hz mode for mode 0Dh, so Keen will run too slowly if synced to vblank. (That might be preferred, but is different enough it probably shouldn't be the default.)
Yeah, that's a great point. Any production ready code will definitely need to be prepared to handle the different environments properly.
clb
Grunt
Posts: 13
Joined: Mon Sep 04, 2023 21:34

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by clb »

Well, looks like I was able to implement smooth 70 Hz updates even without needing to rely on CRT Terminator at all. Everything is running smooth without a single dropped frame in sight.

Although there is definitely an issue that switching from 35fps to 70fps does cause Keen to occassionally miss grabbing a ledge. Maybe there is some kind off odd timing dependent logic there, haven't yet dug into the code.
clb
Grunt
Posts: 13
Joined: Mon Sep 04, 2023 21:34

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by clb »

Huh, curiously, also playing in DOSBox-X is super smooth without frame drops. Although this is running on a 360 Hz ASUS PG259QN display - I wonder if it is aware of > 60 Hz display refresh rates to pace the output better with 70 Hz, not sure how that could work otherwise.

EDIT: Oh yeah, reverting ASUS PG259QN from display control panel from 360 Hz down to 60 Hz does make the Keen code do a microstutterfest. Pretty cool from DOSBox-X that it is able to do that. (I don't happen to have vanilla DOSBox installed right now to figure if that might also do the same)
clb
Grunt
Posts: 13
Joined: Mon Sep 04, 2023 21:34

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by clb »

I pushed my mod up to https://github.com/juj/KEEN70HZ for anyone interested. That was a fun little project. :dopefish
User avatar
K1n9_Duk3
Vorticon Elite
Posts: 757
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by K1n9_Duk3 »

I also noticed the same microstuttering on occasion, especially while playtesting Foray In The Forest (which will run at up to 70 fps by default).

I'm sorry, but I have to report that your version does not always work correctly. I tested your precompiled KEEN70HZ.EXE in DOSBox 0.74.3 with the cycles set to "max" and ran into a constant stream of display glitches while standing on a Bounder in the first level. I was using 70 fps and 1-pixel panning in the game and DOSBox was set to the default "svga_s3" machine type. I suppose this is related to the fact that the CPU speed may change when DOSBox is running in max cycles mode, which would mean your code might also run into problems if the turbo button is toggled while the game is running on a real DOS machine.

I had considered adding your code to FITF when I first read your posts. But given this incompatibilty with max cycles mode, I am going to stick with my old code. The occasional skipped frame is probably easier to tolerate than the display glitches.

By the way, the fact that the refresh manager redraws sprites even if they are completely hidden behind foreground tiles has to do with the fact that the refresh manager assumes all foreground tiles to have transparent parts. Even though the tile images covering the hidden bonus don't actually have any transparent parts in most cases, they are always part of the masked tileset and technically could have transparency. Storing additional information about which foreground tiles have transparency and which ones don't and skipping sprite redraws when the entire sprite is covered by non-transparent foreground tiles was probably not worth the code and data overhead. If redrawing the hidden items was causing performance problems, the easiest solution would have been to add invisible alternative bonus items that will never be drawn at all.

How to fix edge grab and platform riding bugs when running at 70 fps
Open CK_KEEN.C and modify KeenAirThink() like this:

Code: Select all

	if (c.xaxis)
	{
		ob->xdir = c.xaxis;
		AccelerateX(ob, c.xaxis*2, 24);
		
#if (MINTICS < 2)
		// K1n9_Duk3 fix for 70 fps:
		
		if (xtry == 0 && tics < 2)
			xtry = c.xaxis;
			
		// Keen can only grab edges reliably if the hiteast / hitwest values are
		// set correctly, which requires the object to move by at least 1 unit
		// along the x axis. AccelerateX only accelerates every other tic and
		// hitting a wall while jumping always sets xspeed to 0, which means it
		// would take at least 2 tics to make the object move and set hiteast and
		// hitwest to the correct values after hitting a wall. The original Keen
		// games only ran at up to 35 fps (2 tics per frame), so the xtry value
		// would almost never be 0 here. This fix makes sure that Keen always
		// moves by at least 1 unit while jumping and holding down the left or
		// right key, so that Keen can grab edges reliably.
#endif
	}
	else
	{
		FrictionX(ob);
	}
The code snippet above should start at line 1071 in the latest release of my reconstructed code. The #if ... #endif code is what has to be added.

Also in CK_KEEN.C, you need to modify HandleRiding() like this:

Code: Select all

		//
		// The following code aligns the player's position with the platform's
		// position. The screen scrolls in global units (fractions of a pixel),
		// which could cause the player and the platform sprite to move back and
		// forth in relation to each other when the player stands on a platform
		// and the movement causes the screen to scroll. Aligning the positions
		// on the same fraction of on-screen movement avoids this problem.
		//
#if GRMODE == CGAGR
		// Only align player's position with platform position when player didn't
		// move in relation to the platform object:
		
		if (ob->xmove == plat->xmove)
		{
			ob->x &= ~0x3F;
			ob->x |= plat->x & 0x3F;
		}
#else
		// BUG: The original EGA version always applied the alignment, even when
		// the player was walking left/right on the platform, which interfered
		// with the player's movement -- especially in NOPAN mode.
		
		if (ob->xmove == plat->xmove)	// This check was missing in the original code
		{
			// The NOPAN parameter is an undocumented feature in Keen 4-6 that
			// allows the game to run a little less glitchy on systems that don't
			// support pixel panning in hardware (an EXTREMELY rare situation).
			// Sprites can only move in multiples of 8 pixels left/right when the
			// NOPAN parameter was used.
			
			if (nopan)
			{
				ob->x &= ~0x7F;
				ob->x |= plat->x & 0x7F;
			}
			else
			{
				ob->x &= ~0x1F;	// This was missing in the original code
				ob->x |= plat->x & 0x1F;
			}
		}
#endif
The code snippet above should start at line 1913 in the latest release of my reconstructed code. All comments are new.

A few comments about your code
I would have liked a message at the top of any file that you have modified. Section 2 a) of the GPL (v2) says:
You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change.
I'm not sure if that section of the GPL was changed in later versions. Github might track the changes you made, but that information is lost when downloading the code as a ZIP archive. Just to be clear, I'm not asking you to add a full changelog at the beginning of each file you've modified. A simple comment that the file has been changed would be enough in my opinion. That's basically what I did in the ID_* files that I had to modify to recreate the Keen code.

You used the SDL_ prefix for SDL_AlignPreciseTimeCount and SDL_ResetTimeCount. The original naming convention in the ID Engine was that the prefix with the L at the end was used for local routines that were not intended to be used by other parts of the code outside the current "manager". Since those two routines are not used inside ID_SD.C at all, they should use the prefix SD_ instead of SDL_. The Keen 4-6 code does use some of those local routines (MML_UseSpace, RFL_InitAnimList) outside their respective manager files, but that's the exception.

The first smooth-scrolling 70 fps game
I haven't checked the Vogons thread, but I think Dangerous Dave in the Haunted Mansion, Shadow Knights and the first Rescue Rover would be good candidates. These games can run at up to 70 fps, they run in EGA/VGA mode 0Dh (320x200 pixel, 16 colors) and they use screen scrolling via CRTC start and PEL Panning registers. And they predate Keen 4 and even Keen Dreams (in terms of technology and probably also regarding their release dates). In case you didn't know, all of these were written by Id Software in between Keen 1-3 and Keen 4-6 (so definitely in 1991) and were originally published by Softdisk. I'm not exactly sure in which order these games were developed and released, so I can't say which one would have been the first.

It's not always noticeable that these three games run at up to 70 fps because the sprites can only move in multiples of 2 pixels (or 4 pixels in Rescue Rover) left/right, which means some rendered frames are indistinguishable from the previous frame. You need to make the sprites (or the screen) move up/down instead of left/right to actually see the smooth movement.

Keen 1-3, Slordax and the "Dangerous Dave in Copyright Infringement" tech demo were all limited to lower frame rates, so they don't qualify (and the DDICI demo wasn't released to the public until much later).

I guess this means Id Software actually did give "the owners of fast 386/486 PCs something to proudly showcase" in 1991.

However... One needs to keep in mind that any kind of video memory access via the ISA bus was (and still is) a bottleneck. My 386 DX 40 system can't even pull off a constant 35 frames per second in Keen 4. There is a noticeable slowdown when I jump all the way down into the Perilous Pit, for example. 70 fps would be possible, but only when there are only one or two small sprites visible.

John Carmack definitely wasn't the first person to put together CRTC start and PEL Panning for smooth scrolling, though. An article by Michael Abrash was published in Programmer's Journal January/February 1987 that explains these EGA features. The article references a sample program smoothly panning around a virtual 672x384 pixel screen. The PDF version I found of that article doesn't contain source code for the sample program, so I don't know what it actually looked like. It probably wouldn't meet your requirements (the article mentions mode 10h, i.e. 640x350 pixels). But this proves that the basics were known long before Carmack started programming EGA/VGA games.
Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net
clb
Grunt
Posts: 13
Joined: Mon Sep 04, 2023 21:34

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by clb »

Thanks for the detailed and insightful review.

I'll try to incorporate your review points into the repository this coming week.

I tested DOSBox-X in more detail, and I find that on my ASUS PG259QN 360 Hz display, the Full Frame Rate option result is smooth if the display refresh rate is set to >= 120 Hz and cycles are set to around >= 4000 at minimum.

Spent some time trying to figure out why the code struggles in DOSBox-X when my display refresh rate is set to 60 Hz. Tried several different forms, but in the end I realized that even if I degenerate the VW_WaitVBL function into a form

void VW_WaitVBL(int number)
{
ApplyScreenScroll();
long time_end = TimeCount + number;
while(TimeCount < time_end) ;
}

where I keep game loop decoupled from rendering (forcibly set VideoRefreshRateIs70Hz to 0 so that SDL_AlignPreciseTimeCount() does nothing - not that it would be called above anyways)

Then in that test, audio still stutters. I am really puzzled to to understand why that would be possible, since that is absolutely the most minimal form of pacing possible: interrupts are not disabled for even a moment, so audio should be free to pump forward in the audio interrupt. Still, it looks like audio is starving.

Immediately by switching the ASUS display to 120 Hz, audio no longer stutters and results are smooth. But the above code is never even reading the status register 3DAh for any synchronization whatsoever, just waiting in a while( ) loop for enough time to pass. (even if the code hangs in that infinite loop, audio should still tick forward happily)

So DOSBox-X does something naughty, or there is something that I currently misunderstand, I am not quite sure. For now, I pushed an update to README to use at least a >=120 Hz display.

I was thinking that if there is at least some synchronization mechanism that would specifically work on DOSBox, that it would be best to dynamically detect DOSBox and then use a custom code path for that, but given that I got DOSBox to stutter even on the simplest unsynchronized no-interrupts code above, I kind of gave up.
K1n9_Duk3 wrote: Tue Sep 12, 2023 10:42 I suppose this is related to the fact that the CPU speed may change when DOSBox is running in max cycles mode, which would mean your code might also run into problems if the turbo button is toggled while the game is running on a real DOS machine.
I don't think this implication will hold true. The way that a Turbo button works on real DOS machines is that it affects the core multiplier of the CPU. The original Keen 4 game code, and the adjusted sync code measures the EGA/VGA blank signal lengths against the ISA/PCI/AGP bus read speeds. Both of those should stay about the same with the Turbo button, the ISA/PCI/AGP bus does not change speed on Turbo button. There could be minor changes in the speed that a CPU executes instructions in the relevant code, but the bus read speeds should still dominate.

This was a specific concern on my mind as well, and I did verify that was the case on my 486 system where I toggle between 80 MHz and 25 MHz.

On DOSBox though changing the cycles settings after startup could/would/might have an effect, all bets are off. Which is why using a custom vblank code for DOSBox would make sense, if one was possible.

Before settling on the current version, the first version of the hblank/vblank sync code I wrote did have a self-calibration where at the start wait on each frame, the hblank length was measured. But doing such autocalibration at each vblank wait will require disabling interrupts for a longer while, which is why I settled on pre-calibrating at startup as a more reliable option.
K1n9_Duk3 wrote: Tue Sep 12, 2023 10:42 If redrawing the hidden items was causing performance problems, the easiest solution would have been to add invisible alternative bonus items that will never be drawn at all.
Yeah, that was my thinking. "why didn't they have the game level artists flag hidden candies as hidden...". Though maybe this game never was like Doom where optimizing every possible opportunity was important. Although I think there is a lot of untapped potential in the code to make it work smoother even on an XT.

I will look at the other things in the coming week, and to make the code GPL compliant for you.
clb
Grunt
Posts: 13
Joined: Mon Sep 04, 2023 21:34

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by clb »

Oh one more note on behavior of DOSBox-X:

they currently implement the S3 Trio SVGA as default, but they implement the scroll register latching different from the real S3 card.

On a real S3 Trio, latching occurs as follows: (from https://www.vogons.org/viewtopic.php?p=1190977#p1190977)

Display Start Address register: end of vsync
Horizontal Pixel Shift Count register: start of vsync

but DOSBox-X implements the latching as follows:

Display Start Address register: *twice per frame*, first at vblank start, and second time at vsync start
Horizontal Pixel Shift Count register: *twice per frame*, first at vblank start, and second time at vsync end

The horizontal scroll glitch occurs if the registers are reprogrammed between these two events, so that one makes it into the next frame, but the other doesn't.

So for DOSBox-X, proper scroll sync code will either program the register during visible picture area (pretty easy to do without needing to disable interrupts even), or after vblank start but before vsync start (this is what original Keen4-6 did, and what my changed version of the sync code also does, but it is awkward with respect to interrupts), or after vsync end.

But overall, looks like the issue with DOSBox-X audio starving is somewhere else than the vertical sync code.
User avatar
K1n9_Duk3
Vorticon Elite
Posts: 757
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by K1n9_Duk3 »

About the audio stuttering you reported when setting your display to 60 Hz:
I think this is caused by DOSBox itself and has (alomst) nothing to do with the code you are trying to run in DOSBox. It's just a side effect of your display driver forcing DOSBox to refresh at 60 Hz instead of 70 Hz or whatever rate DOSBox is trying to use. I ran into that same problem years ago when I tried to use Fraps to record DOSBox footage. The version of Fraps that I used limited the frame rate to the desired video frame rate (I think it was 60 fps) and that caused severe audio stuttering in DOSBox while recording.

Fraps also showed me that DOSBox doesn't render any new frames as long as the displayed image doesn't change. So if the game itself runs at less than 60 fps, the chances of DOSBox being blocked by an external frame limiter are minimal.

In my experience, DOSBox usually runs fine on a 60 Hz display. I've been using 60 Hz displays for over a decade, and I've never had any audio stuttering in DOSBox (except while recording video Footage with Fraps). That's why I think that when you set your display rate to 60 Hz, this acts as a frame limiter and prevents DOSBox from running as intended.
Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net
User avatar
K1n9_Duk3
Vorticon Elite
Posts: 757
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

Re: Anyone know what could cause the microstuttering in Keen 4-6?

Post by K1n9_Duk3 »

I found a potential bug in your 70 Hz code. In BENCHVS.CPP, you use the outp() function to write byte values to a port. Borland introduced a bug in Borland C++ 3.1 that will break your code. The instruction that will cause problems is the following line in SetTimerIntr():

Code: Select all

  outp(0x40, interval >> 8);
In Borland C++ 3.1, DOS.H (and also CONIO.H) declare outp() like this:

Code: Select all

#define outp(__portid, __value)     __outportb__(__portid, (unsigned char)__value)
So the preprocessor would end up turning your code into this:

Code: Select all

  __outportb__(0x40, (unsigned char)interval >> 8);
In case it isn't obvious, the typecast limits the "interval" value to 8 bits BEFORE the shift operation is performed, which effectively causes the code to write a zero to port 0x40 instead of writing the high byte of the interval variable.

The solution would be to use outportb() instead of outp() or to wrap the second parameter in parentheses.

You also mixed both inp() and inportb() in BENCHVS.CPP, so I would suggest replacing outp() with outportb() and inp() with inportb() to avoid the potential issue in Borland C++ 3.1 while also making the code more consistent. The ID Engine always uses inportb() and outportb() for byte-sized port reads and writes.
Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net
Post Reply