The Great Galaxy-Mod Bug Hunt 2020

Here is where to post about the latest Commander Keen fangame or modification you've finished, a new website you've made, or another Keen-related creation.
User avatar
K1n9_Duk3
Vorticon Elite
Posts: 781
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

The Great Galaxy-Mod Bug Hunt 2020

Post by K1n9_Duk3 »

Just to get this out of the way first: When I refer to Galaxy-Mods, I'm talking about all the mods for Keen 4-6 as well as Keen Dreams, not just for the two games that were actually part of the "Goodbye, Galaxy!" series.

As some of you might already know, I started digging deep into the realm of Galaxy-Mods this year. My initial motivation being the seemingly random crashes that some of you might have run into while playing some of the Galaxy-Mods.

Some of the bugfixes I came up with address general issues that already existed in the original Keen games, the mods merely inherited these bugs from the games they were based on. Other bugfixes address issues that were introduced by the mod's patches, like miscalculated jump addresses and other errors that resulted in memory corruption when the patched code was executed. In some cases, some of the files were incomplete (mostly EGAHEAD files), indicating that the tools used to edit the EGAGRAPH files weren't working corretly. And in one case (MAPHEAD data in the "official" release of Keen Meets the Meats) the data was outdated, preventing the game from loading most of the levels at all.

My initial goal was to fix the parts of each mod's patch file that needed to fixed, but I ended up doing a little more than just that. For some mods, I also edited some of the levels to make sure that there are no holes in the walls that Keen could climb into and either get stuck in the wall or glitch out and get pushed through solid walls.

But I have to admit, I haven't actually played through half of these mods, so there might be things that I've missed. This is where all of you can help. Here are the current versions of all the mods that I've "fixed" so far:

Keys of Krodacia
Dead in the Desert
Battle of the Brains
Keen Dreams Plus

The Eight Accumulators (Update: 2021-01-22)
Suburb Shenanigans (Update: 2020-11-25)
Operation Ocflore (Update: 2020-11-25)

The Terror From Outer Space
Marooned on Mars - Galaxy Version

XMAS 2010
Keen Meets the Meats
Dead in the Desert - Community Mappack

Aliens Ate My Babysitter Again
Bounty Hunting Brouhaha
Hello, Galaxy!
Mink's Puzzle Pack
Short Circuit Oracle
The Sunnylands
Mirror Menace (Update: 2020-12-30)

Underworld Ultimate (Update: 2021-01-06)

Please playtest them and if you find any bugs, please post them in this thread. If it's something I can fix (without having to redesign the entire game), then I'll fix it and update the download links.

Please note that you should not try to load saved games that were saved in older versions of these mods. That's because the old saved games might already have some corrupted data in them (if there were bugs in the old patches that caused memory corruption). And in some cases, I had to move some of the data structures around to avoid errors. Old saved games might look for these data structures at the old position and therefore will not work with the new patches in place. So please don't use old savegames.


Appendix:
Since I'm probably not going to release an update for every mod ever made (for various reasons), I figured I should probably share the results of my investigation and let you know which mods are fine and which ones have issues in the patch scripts. Just to be clear, I'm always referring to the most recent version on the KeenWiki (as of August 2020).

The following mods have (sometimes severe) issues, but I probably won't create any updates for them:
- The Prisoner's Dilemma
- Galaxy Defender
- The Temple of Cydonia
- Adventures in the BWB Megarocket
- Cyber Zero's Episode -3
- Zero X
- BWB Megarocket Keen 5
- The Grand Intelligence Gold
- Intelligent Intellect
- Another Helping of Veggies
- Keen 4 "Ultimate Edition"
- Space Evaders
- Kube
- Pakeen
- Kreeg Crisis
- Keen Dreams Modette

The following mods also have severe issues, but should get an (unofficial) update at some point:
- Keen's Valentine Bash
- Skater Squirrel
- Zoltan's Revenge

Mods that don't seem to have any patching bugs:
- Keen 5.5
- The Armageddon App
- Torgopolis
- Birdman
- Gridlock's abandoned mods (Keen 1 redone, Lava)
- Commander Keen x BioMenace
- Pyramid of the Allowed
- any mod or level pack that only patches in the new audiohed/egahead/maphead data (obviously)


Edits:
2020-11-25: added Operation Ocflore and updated Suburb Shenanigans.
2020-12-30: added Mirror Menace and Underworld Ultimate.
2021-01-06: updated Underworld Ultimate.
2021-01-10: added the Appendix section
2021-01-24: added update note for The Eight Accumulators
Last edited by K1n9_Duk3 on Sun Jan 24, 2021 22:50, edited 5 times in total.
Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net
User avatar
Roobar
Vorticon Elite
Posts: 3267
Joined: Tue Jan 08, 2008 16:12
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by Roobar »

In Suburb Shenanigans I got this glitch: https://youtu.be/25BTzrGVlRc
It's one of the early levels next to the spot house.

Other than that, I didn't found any bugs or crashes.
User avatar
Nisaba
Janitress
Posts: 1597
Joined: Fri Jan 01, 2016 23:34
Location: The Outpost
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by Nisaba »

farmer-dancing-at-the-pool-side. he seems to be quite a bit dithery, don't you think. maybe he's trying to scare away 'em aquatic bees...

on a more serious note:
this is an insane endeavor. what you seem to have achieved here is simply mind blowing. Debugging must have taken countless hours. what an incredible approach. Kudos!

with this major contribution to the community we already have a very strong candidate to win the trophy " Keenest Person of the Year".

gonna upload those fixed version to the wiki at some point. But as you've already mentioned let's just test and dig for possible bugs and inconsistencies.
out now (link) : Image
User avatar
Syllypryde
Vorticon Elite
Posts: 1025
Joined: Tue Jan 20, 2009 18:33
Location: Michigan
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by Syllypryde »

I've played through every single one of these games. The funny thing is I didn't even realize most of them had any bugs. I might be apt to replaying them if I knew what it was you "fixed" in each one. Right now I just don't have the time to completely play through each of these games just to find a fix or two. But all in all, it is really cool you took the time do all this. This was definitely appreciated.
.niarb ym fo
snoitulovnoc eht tuaba
selbbarcs ssensuoicsnoc
rehgih a ekil smees
Benvolio
Vorticon Elite
Posts: 976
Joined: Sun May 29, 2011 12:43
Location: Ireland
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by Benvolio »

This project you have done is huge, so thank you for your effort! However, like Syllypryde, I would not say that significant bugs have been a feature of my experience playing any of the main Galaxy releases over the years. Have you any stand out examples of commonly experienced problems in any of the big mods (Keen9, Atroxian, Ocflore etc)?

What would interest me would be an effort to certify Keen mods as to how well they run on various eras of old hardware. My experience with old PCs is narrow in its scope, but some of the classic early Keen1 and Keen2 mods played perfectly on hardware dating from 1986. But it would be interesting if someone certified mods (vorticons, dreams and galaxy) for how they run on say 8086 era systems, 286, 386, 486 etc. Probably not crucial work as who doesn't have dosbox on a late 2010s machine, but interesting nonetheless.
User avatar
K1n9_Duk3
Vorticon Elite
Posts: 781
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by K1n9_Duk3 »

I just added new version of Ocflore and Suburbs.

The update for Suburb Shenanigans fixes the bug that Roobar found (the Gardener's hitbox was different for one sprite, causing him to float and get stuck)

The Ocflore update fixes a problem with the hitbox of the Enkri Meager (brown flying thing) that coud cause them to get moved through walls when turning around in tight spaces. Xrack stumbled upon that bug during one of her streams. I also added a patch that should prevent the berkeloid-replacements from getting stuck after shooting (this bug also popped up several times during Xrack's playthrough).
Syllypryde wrote: Wed Nov 25, 2020 18:18 I might be apt to replaying them if I knew what it was you "fixed" in each one.
Each of these downloads includes a changelog.txt file.

The most important changes in these unofficial bugfixes are the ones that fix patches that are just flat out wrong (miscalculated jump addresses, using the wrong registers etc.) These are the bugs that will inevitably cause some form of memory corruption when the code is executed. This memory corruption might be hamless if it affects a memory region that is currently unused, but you might also get corrupted level objects that will cause the game to crash as soon as that level object becomes visible on the screen. If you saved the game that corruption might also be part of the saved game, meaning that the game will always crash when you load that saved game. It might not crash right away, once the corrupted object becomes active, it will cause a crash.

Most mods for Keen 5 and Keen 6 also have a very common source for errors. The teleporters on the world map (and also the elevator shaft in Keen 5) store the destination coordinates in the infoplane. Level entrances are also stored in the infoplane. When you return to the world map, the game removes the level entrances for the levels that have been completed. But due to the way this was coded, this would also remove the destination coordinates from the world map if the y coordinate matches the level number of a completed level. If you have ever seen the game scrolling to the top left corner of the map when entering the elevator shaft, or just locking up completely when entering a teleporter (as seen in the old versions of "Marooned on Mars - Galaxy" and "Mirror Menace"), this is why that happened.

Most of the generic fixes I added for every mod are quality of life improvements, like my screen update code. Others are there to make it less likely for the game to run out of memory and crash. Memory issues like this are (usually) few and far between when playing in DOSBox (since most modders playtest their mods in DOSBox), but it's nice to have a pretty robust code base when you move away from DOSBox and run these mods on real DOS PCs with a less than optimal memory configuration.

I remember that the initial release of Suburb Shenanigans had massive problems when it came to memory. I think Nisaba changed the layout of some of the levels to reduce the memory requirements for v1.1 of the mod. And even with these changes, the game still needed at least 476k to play all of the levels with music enabled. In reality, it might have needed even more than that once the memory became too fragmented. When you play the mod in DOSBox via CKPATCH, there can't be more than 490k available to the game, leaving only a 14k "memory reserve" for the situations where the memory becomes too fragmented to make efficient use of what's available. This usually meant that you'd get an "insuficient memory for background music" message when entering some of the levels, but it might also have caused the game to crash. With this update, the 476k requirement has been reduced to 457k, which leaves a "reserve" of about 33k when playing the new version of Suburb Shenanigans via CKPATCH in DOSBox.
Benvolio wrote: Wed Nov 25, 2020 20:44 What would interest me would be an effort to certify Keen mods as to how well they run on various eras of old hardware.
I kind of did some work that might be useful when trying to get these mods to run on true DOS systems. The changelog.txt files included in each archive also lists the minimum memory requirements for each mod (both with and without music). While this doesn't say much about the performance side of things, the memory requirements will be a huge hurdle for any pre-286 system with less than 1 megabyte of RAM.

I included the memory requirements because I actually played most of these mods on a pure DOS laptop (Pentium MMX) and I also have a 386 PC (and a 286 motherboard) which I sometimes use to playtest Keen mods.

I'm not sure if it would be possible to play mods on a system with an 8088 or 8086 CPU. While the original Keens were using only 8088/8086-compatible instructions, the CKPATCH program might be targeted at 286 or 386 systems. Another thing you would need to keep in mind is that some mod's patches use 286 opcodes (because they're shorter/smaller than the equivalent 8086 code). If a mod uses any 286 code, it may seem to run fine on an 8086 at first, but once the game tries to run the 286 code, it will crash the system.
Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net
User avatar
Nisaba
Janitress
Posts: 1597
Joined: Fri Jan 01, 2016 23:34
Location: The Outpost
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by Nisaba »

K1n9_Duk3 wrote: Wed Nov 25, 2020 23:06
Benvolio wrote: Wed Nov 25, 2020 20:44 What would interest me would be an effort to certify Keen mods as to how well they run on various eras of old hardware.
I kind of did some work that might be useful when trying to get these mods to run on true DOS systems. The changelog.txt files included in each archive also lists the minimum memory requirements for each mod (both with and without music). While this doesn't say much about the performance side of things, the memory requirements will be a huge hurdle for any pre-286 system with less than 1 megabyte of RAM.

I included the memory requirements because I actually played most of these mods on a pure DOS laptop (Pentium MMX) and I also have a 386 PC (and a 286 motherboard) which I sometimes use to playtest Keen mods.

I'm not sure if it would be possible to play mods on a system with an 8088 or 8086 CPU. While the original Keens were using only 8088/8086-compatible instructions, the CKPATCH program might be targeted at 286 or 386 systems. Another thing you would need to keep in mind is that some mod's patches use 286 opcodes (because they're shorter/smaller than the equivalent 8086 code). If a mod uses any 286 code, it may seem to run fine on an 8086 at first, but once the game tries to run the 286 code, it will crash the system.
@keenmaster486 might be able to do some dedicated hardware testings...
out now (link) : Image
Benvolio
Vorticon Elite
Posts: 976
Joined: Sun May 29, 2011 12:43
Location: Ireland
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by Benvolio »

When I had an 8086 I recall playing Norp the Yorp 2 on it.
User avatar
keenmaster486
Vorticon Elite
Posts: 542
Joined: Sun Mar 20, 2016 18:29
Location: Tranquility Base
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by keenmaster486 »

Nisaba wrote: Mon Dec 14, 2020 10:13 @keenmaster486 might be able to do some dedicated hardware testings...
That is correct. I can test on real DOS machines including a 286. What do you need me to test?
I flermmed the plootash just like you asked.
User avatar
Nisaba
Janitress
Posts: 1597
Joined: Fri Jan 01, 2016 23:34
Location: The Outpost
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by Nisaba »

keenmaster486 wrote: Mon Dec 14, 2020 23:17 That is correct. I can test on real DOS machines including a 286. What do you need me to test?
here's a TL:DR summery:
K1n9_Duk3 worked his way through our major Galaxy-Mods and fixed most if not all patching cruelties. As this required some heavy lifting we were wondering whether those updates do have an impact in one way or another, if tested against real hardware from back in the day. this is where I was thinking about you, as you are, so to speak, the communities hardware dude.

In a nutshell, this is what needs to get tested:
- do you encounter the mods to run out of memory and crash while playing on real DOS PC's?
- what about the throughout performance?

you can find all links to the mods in questions in the first post of this thread
EDIT: if I had to pick one mod for a start, I'd choose "Suburb Shenanigans"


further reading:
K1n9_Duk3 wrote: Wed Nov 25, 2020 23:06 I kind of did some work that might be useful when trying to get these mods to run on true DOS systems. The changelog.txt files included in each archive also lists the minimum memory requirements for each mod (both with and without music). While this doesn't say much about the performance side of things, the memory requirements will be a huge hurdle for any pre-286 system with less than 1 megabyte of RAM.

I included the memory requirements because I actually played most of these mods on a pure DOS laptop (Pentium MMX) and I also have a 386 PC (and a 286 motherboard) which I sometimes use to playtest Keen mods.

I'm not sure if it would be possible to play mods on a system with an 8088 or 8086 CPU. While the original Keens were using only 8088/8086-compatible instructions, the CKPATCH program might be targeted at 286 or 386 systems. Another thing you would need to keep in mind is that some mod's patches use 286 opcodes (because they're shorter/smaller than the equivalent 8086 code). If a mod uses any 286 code, it may seem to run fine on an 8086 at first, but once the game tries to run the 286 code, it will crash the system.
Last edited by Nisaba on Wed Dec 16, 2020 21:44, edited 1 time in total.
out now (link) : Image
User avatar
keenmaster486
Vorticon Elite
Posts: 542
Joined: Sun Mar 20, 2016 18:29
Location: Tranquility Base
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by keenmaster486 »

Great! I will try to get to that this week.

A quick note, CKPATCH seems to work fine on a 286 for me.

I do not have an 8088 or 8086 machine to test with, though.
I flermmed the plootash just like you asked.
User avatar
K1n9_Duk3
Vorticon Elite
Posts: 781
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by K1n9_Duk3 »

Just in case the probem ever arises, you should be able to use version 2.0 of my patching utility to create fully-patched executables from almost any CKPATCH script. You don't even have to modify the patch script anymore, it should just work out of the box.

The exception are mods that require the TSR part of CKPATCH to redirect certain file names to other files, mostly used for renaming the level files in Keen 1-3 mods. Keen 4-6 mods may also use this feature to rename the EGAGRAPH, AUDIO or GAMEMAPS files, but my utility will just patch the file name in the executable, assuming the new file name fits in the available space. I haven't seen any Keen 4-6 mods where this would be an issue.

Using the fully-patched executables will give you some more free memory and you could also use them to run mods on systems that can't run CKPATCH. I know that at least some versions of the CKPATCH executables themselves are compressed and I think I read somewhere that the decompression code might be incompatible with 8088 and 8086 machines. However, if the patch scripts insert any 286 instructions, the mod would still crash on an 8088/8086 machine whenever the CPU tries to execute that code.
Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net
User avatar
DarkAle
Vorticon Elite
Posts: 318
Joined: Wed Jul 26, 2017 17:40
Location: Coral Hideout
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by DarkAle »

Hey there,
I tested The Sunnylands a few days ago and here's what I think about this bugfixing: GREAT JOB! I played the game at normal and I didn't find ANY bug! Hope you can fix the bugs in Valentine's Keen, too!
Commander Keen in... Robot Apocalypse!
Episode 13: The Ultra Omegamatic viewtopic.php?f=4&t=6802
Episode 14: Secret of the Sorcerer viewtopic.php?t=13031
User avatar
K1n9_Duk3
Vorticon Elite
Posts: 781
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by K1n9_Duk3 »

Thank you for the feedback. To be honest, The Sunnylands didn't really have bugs, it just used the wrong scrolling type in one of the underwater levels. The remaining changes are mostly just generic fixes to avoid running out of memory or other memory-related errors when playing on a system that has barely enough free memory to play this mod (real DOS PCs and such).
DarkAle wrote: Fri Dec 18, 2020 19:32Hope you can fix the bugs in Valentine's Keen, too!
Yeah, I'm still working on that one. I think I already fixed all of the bugs that would have caused crashes. That mod has some additional problems that are beyond the scope of my little bugfix project. I think that the dark cave level near the end is completely unfair and should definitely not be a level that you absolutely have to complete in order to finish the game. I'm tempted to swap it with one of the (currently optional) cave levels. Aside from that, I still need an idea how to make the puzzle mechanics that are introduced in the Castle Ramenstein level a little easier to understand for first time players.
Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net
User avatar
K1n9_Duk3
Vorticon Elite
Posts: 781
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

Re: The Great Galaxy-Mod Bug Hunt 2020

Post by K1n9_Duk3 »

I just added Mirror Menace and Underworld Ultimate.

Underworld Ultimate only had its crippling bugs fixed. The difficulty i still the same, although the changes I made to the scrolling while Keen is floating should make some areas a little less frustrating. I haven't actually finished my playthrough, so if anybody can point out specific areas and/or enemies that need to be changed to make this mod a bit more enjoyable, let me know!

By the way, the bugfixed version of Underworld Ultimate finally allows you to collect all the booze bottles. In the old version, you actually "lose" the bottle from level 3 once the base appears on the world map, and the final booze bottle can only be obtained after the base appears.

Keep in mind that old saved games are incompatible with this update!


The Mirror Menace update is a bit unusual compared to the other updates in this thread. I actually broke my own rule here and changed the level designs a lot more in order to eliminate all parts that could lead to the player being stuck in an unwinnable state in the level. And I also changed a couple of other things in an attempt to make the levels a little less unfair to first-time players. I don't really consider myself a level designer, but I hope I didn't make things worse than they were before.

Again, old saved games are incompatible with this update!
Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net
Post Reply