Foray in the Forest - now available as a patch-based mod for Keen 4

Here is where to post about the latest Commander Keen fangame or modification you've finished, a new website you've made, or another Keen-related creation.
Post Reply
User avatar
K1n9_Duk3
Vorticon Elite
Posts: 786
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

Foray in the Forest - now available as a patch-based mod for Keen 4

Post by K1n9_Duk3 »

Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net
User avatar
Quillax
Vorticon Elite
Posts: 572
Joined: Sat Mar 14, 2015 14:41
Location: Quillax Ship
Contact:

Re: Foray in the Forest - now available as a patch-based mod for Keen 4

Post by Quillax »

At first, I thought this had to be a late April Fools' joke. Then I downloaded it, and...

:O

K1n9_Duk3, you have really impressed me! My jaw actually fell when I was playing this for a bit! All this time, I thought Foray in the Forest would be impossible to make as a patch-based mod, and you proved me dead wrong! Is there really no difference between patch modding and source modding!? Words simply cannot describe this incredible achievement...
Quillax Ship - Where you can download all of my cool, keen mods and super-marvelous fangames!
NY00123
Vorticon Elite
Posts: 511
Joined: Sun Sep 06, 2009 19:36

Re: Foray in the Forest - now available as a patch-based mod for Keen 4

Post by NY00123 »

Even before downloading, I've had a guess of how may it be possible, technically speaking.
Briefly checked the PAT file (just a few seconds), it's more-or-less as suspected by me. Haven't done more than that, but I may let others write here if anyone wants to.
User avatar
Nisaba
Janitress
Posts: 1612
Joined: Fri Jan 01, 2016 23:34
Location: The Outpost
Contact:

Re: Foray in the Forest - now available as a patch-based mod for Keen 4

Post by Nisaba »

.
out now (link) : Image
User avatar
K1n9_Duk3
Vorticon Elite
Posts: 786
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

Re: Foray in the Forest - now available as a patch-based mod for Keen 4

Post by K1n9_Duk3 »

This patch-mod thing is really just a proof-of-concept. This monster of a patch file literally just patches every single byte of the Foray in the Forest v1.0 code into Keen 4, overwriting pretty much all of the original Keen 4 code. I didn't rewrite the whole FITF code in patch form, I just wrote a program that turns the FITF executable into a patch script. The patch-mod version is identical to the regular release version (except for a slightly reduced amount of available memory due to running the game via CK4PATCH). Saved games and the config file are 100% compatible.

What I wanted to prove is that you can really patch anything into the code, as long as the old code provides enough space for whatever you want to patch in.

If you want, you can take the MONSTER.PAT file, edit the %ext line (and delete the %version line) at the beginning and turn FITF into a mod for Keen 4 CGA, Keen 5, Keen 5 CGA, Keen 6, Keen 6 CGA, Keen Dreams (EGA v1.13 only), Dangerous Dave 2, Dangerous Dave 3, or any episode of BioMenace, as long as you have access to the game's executable and a CKPATCH utility that supports that executable.

I have only tested the patch-mod version with Keen 4-6 (v1.4 EGA) and Dreams (v1.13 EGA). I have not tested the CGA versions or any of the other games (Dave 2-3, BioMenace 1-3). Out of all the games listed in the previous paragraph, Keen Dreams 1.13 is the one providing the least amount of space to insert new code. Since Keen Dreams seems to work fine as a base for patching in the FITF code, I assume that the other games would work as well.

The main reason why FITF "fits" into Keen 4-6 is that those games had other files like EGAHEAD, MAPHEAD and so on linked into the executable. FITF loads them from external files after the program is started. Another contributing factor is that the uninitialized data ("BSS" segment and initial stack) in these games is part of the executable image, which provides even more space for patching in new code. Combined with the fact that FITF uses 286 instructions and compiler optimizations that prefer smaller code over faster execution, this makes the FITF executable smaller than most of the original executables from these games. So even though there is a lot more code in FITF, the old executables still provide enough space for the FITF code.

To be honest, I didn't even know that the FITF code can actually fit into Keen Dreams EGA v1.13 (not the CGA version 1.05 though!) before I started writing this post. I knew that Keen 4, 5 and 6 would work, but I didn't know that Dreams (and the other games) could also work. It probably would have made for an even more interesting title for this thread.

As you can see from the time stamp of MONSTER.PAT, this patch-mod version is not an entirely new thing. It even predates the recent Are "source code mods" mods or fangames? discussion.

It all started with me beginning to write a response to Quillax's first post in the FITF release thread. Parts of that message were about what can be patched into existing code and how easy or difficult patching the code could be. As I was writing that message, I kept questioning my own statements to make sure what I wrote was objectively true. One of my conclusions was that you can patch any new code into any game/program as long as there is enough space for the new code. After I wrote that statement, I looked at the size of FITF.EXE and the size of the uncompressed KEEN4E.EXE and thought that this could actually work. So I dug a little deeper and eventually started to write a little program that would generate a patch script for patching the FITF code into Keen 4 to see if this would actually work correctly.

Creating the patch file was relatively easy. Getting it to work with CK4PATCH was the bigger problem. I kept getting "Can't open EGAGRAPH.CKS!" errors when using CK4PATCH. But generating a fully-patched executable with my patching utility worked fine. As it turned out, CK4PATCH redirects any open file commands for any files that start with "AUDIO", "EGAGRAPH" and "GAMEMAPS" to the default Keen 4 files "AUDIO.CK4", "EGAGRPAH.CK4" and "GAMEMAPS.CK4" even if the patch file doesn't use the %audio, %egagraph, or %gamemaps instructions to rename these files and such redirects would not even be necessary in the first place. Renaming FITF's resource files into something that doesn't trigger the redirections allowed me to work around that problem and finally get it to work with CK4PATCH.

My original idea was much more convoluted than what's now been released. I was splitting the FITF code into two parts to make sure the start of the data segment in my patched-in FITF code would be the same amount of bytes away from the initial stack pointer in Keen 4 as it was in the real FITF executable. That would explain why Keen Dreams wasn't on my radar (this approach would not have worked for Keen Dreams). But after examining the startup code more closely, I realized that it didn't matter what the initial stack pointer was, since it would be replaced pretty early on in the startup code anyway.

So, yeah. This is a proof-of-concept. There may be bugs in these patch-mod versions that I'm not aware of. You can play the game like this, but using the real FITF.EXE is definitely the better option.
Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net
User avatar
Syllypryde
Vorticon Elite
Posts: 1039
Joined: Tue Jan 20, 2009 18:33
Location: Michigan
Contact:

Re: Foray in the Forest - now available as a patch-based mod for Keen 4

Post by Syllypryde »

K1n9_Duk3 wrote: Sat Apr 06, 2024 8:04 This patch-mod thing is really just a proof-of-concept........
That it is. I've already played and completed your patched mod version of FITF on normal and it worked just fine without a single error or bug. Maybe now this will silence all the people that act as if those who created source code mods are somehow creative hacks and untalented bums just because they somehow believe that working within a superficial set of limitations makes you a fraud if you choose not to allow your creativity to be limited. They seem to disregard the fact that nearly every modder in this community, most especially the source code modders have been working within those limitations for 10-15+ years and have more than proven their creativity. In fact 2 of those 4 source mod modders have created some of the best, if not the best mods in the community.

All the people screaming "It's not a mod!" seem to have forgotten the meaning of what modding was supposed to be in the first place. The purpose of modding was to not only capture the magic in the original games, but to also take those games to the next logical steps. When the technology for modding evolved, it's natural that modders would use that technology to create even more amazing mods. They have all more than proven that they can create good games within the limitations that were originally mandatory. Now that they no longer have to have their creativity limited anymore, they can now make the games they had envisioned and thought was never possible. As long as the things that made us all love Commander Keen are still intact, why should it matter exactly how the game was created?

One of the arguments being made is that changing a few of the game mechanics and adding in features not originally in the official games means they are no longer considered mods. Seriously? According to this logic doesn't that make most of Keenrush's games no longer mods? In most of his mods he has changed the game mechanics so radically that the games are barely recognizable to the original games, though still using Keen as the main protagonist. So according to this logic Keenrush's creations are no longer mods. There are others that have created games that have little to nothing to do with Commander Keen. Examples include:

---Marta
---Martinez
---Bananasaurus
---Character
---Mario
---Space Invaders
---Norp the Yorp
---Cube
---PacKeen
---Birdman
---Monochrome
etc.

So according to this logic these are no longer mods either. Yet somehow they are and so are Keenrush's mods. So if changing game mechanics, changing the protagonist and changing the theme of the game where it has little or nothing to do with Keen are no longer mods than why the hell are all of these mods, yet source code mods that stay true to the spirit of Commander Keen far more than most of the aforementioned games do are somehow no longer mods? Seriously? Sounds like to me a few select people are imposing arbitrary, cherrypicked, superficial, and most of all hypocritical limitations onto the modders.

Speaking of hypocrisy, most of these same people throwing out their opinions of what is not a mod are the same people for DECADES have saying AD NAUSEUM that Commander Keen is outdated:

---Dying in one hit, a health system should be used
---Loading and saving, checkpoints should be used
---extra lives are meaningless
---score is meaningless
---offers no meaningful rewards
--- most of all Keen needs to start using more game mechanics that modern video games use

I hate to have to be the one to tell you this, but these modern day game mechanics have been created for games played on modern game consoles, not on a mac or a PC. Most if not all of these mechanics are impossible for a DOS game on a PC. The same people that want modders to operate within superficial limitations of the original game engine and if they don't they are not mods, but in the same breath think that Keen should adapt the game mechanics of modern video games created by billion dollar companies, and somehow they will be mods. Yet a modder incorporating some of these game mechanics you demand into a mod, but use a source code to do it... NOPE! It's not a mod!!! Seriously?

But now that K1n9_Duk3 has proven that a source code can operate efficiently under the limitations of a patched mod with FITF, this should put the legitimacy of source code modding to rest once and for all. I have to side with K1n9_Duk3, Nisaba, Gridlock, Ceilick, and DarkAle etc. on this debate. As long as the things that made us all love Commander Keen are still intact, why should it matter exactly how the game was created?

This is my 95 cents on this discussion. Why 95 cents instead of 2 cents? Inflation....
.niarb ym fo
snoitulovnoc eht tuaba
selbbarcs ssensuoicsnoc
rehgih a ekil smees
User avatar
Quillax
Vorticon Elite
Posts: 572
Joined: Sat Mar 14, 2015 14:41
Location: Quillax Ship
Contact:

Re: Foray in the Forest - now available as a patch-based mod for Keen 4

Post by Quillax »

K1n9_Duk3 wrote: Sat Apr 06, 2024 8:04 This patch-mod thing is really just a proof-of-concept. This monster of a patch file literally just patches every single byte of the Foray in the Forest v1.0 code into Keen 4, overwriting pretty much all of the original Keen 4 code. I didn't rewrite the whole FITF code in patch form, I just wrote a program that turns the FITF executable into a patch script. The patch-mod version is identical to the regular release version (except for a slightly reduced amount of available memory due to running the game via CK4PATCH). Saved games and the config file are 100% compatible.
Yeah, I looked at the patch file after playing around with the mod for a bit, and I suspected that you just simply copied the compiled machine code (in hex form) from FITF and applied it as patches that overwrite the whole game, which seems to be close to what you actually did. The fact that the patch file can work with other Keen games (like Keen Dreams, which really surprised me although I haven't tried it yet) seems to confirm the overwriting part.
K1n9_Duk3 wrote: Sat Apr 06, 2024 8:04 As you can see from the time stamp of MONSTER.PAT, this patch-mod version is not an entirely new thing. It even predates the recent Are "source code mods" mods or fangames? discussion.

It all started with me beginning to write a response to Quillax's first post in the FITF release thread. Parts of that message were about what can be patched into existing code and how easy or difficult patching the code could be. As I was writing that message, I kept questioning my own statements to make sure what I wrote was objectively true. One of my conclusions was that you can patch any new code into any game/program as long as there is enough space for the new code. After I wrote that statement, I looked at the size of FITF.EXE and the size of the uncompressed KEEN4E.EXE and thought that this could actually work. So I dug a little deeper and eventually started to write a little program that would generate a patch script for patching the FITF code into Keen 4 to see if this would actually work correctly.
Ah, I didn't even look at the time stamp; I thought this was a response to that recent debate over source mods (I've planned on saying some things at some point).

I'd like to thank you for creating this proof-of-concept, it really helped me understand that source modding is closer to patch modding than I had initially thought all this time. I thought that the EXE files of source mods were allowed to much bigger and thus have more code since they're compiled, in fact FITF.EXE appears to be twice as large as KEEN4E.EXE which is now making me raise an eyebrow (is the latter smaller because it has more empty space?). Though, looking at the patch file, it seems that you created a lot of empty space; is that taken up by data from external files (like the strings file) or is it really unused space, meaning that FITF can be even bigger than it is right now?

I thought that source modding was kinda like GZDoom where you didn't have to worry about memory limitations so much, especially if you're developing for Omnispeak, but the reality appears to be that when programming for DOS you still have to deal with tight limitations.
Syllypryde wrote: Sat Apr 06, 2024 10:36 That it is. I've already played and completed your patched mod version of FITF on normal and it worked just fine without a single error or bug. Maybe now this will silence all the people that act as if those who created source code mods are somehow creative hacks and untalented bums just because they somehow believe that working within a superficial set of limitations makes you a fraud if you choose not to allow your creativity to be limited. They seem to disregard the fact that nearly every modder in this community, most especially the source code modders have been working within those limitations for 10-15+ years and have more than proven their creativity. In fact 2 of those 4 source mod modders have created some of the best, if not the best mods in the community.
One of the things that appealed to me about mods, both in creation and playing, is working with the limitations set by the game's structure, creatures, etc. and seeing how far they can be pushed, which often leads to creativity. Though I quite enjoy heavily patched mods like KeenRush's stuff, I often think about how the authors had to carefully decide which parts of the game to overwrite and which parts to reuse. I might go into more detail about that.
Syllypryde wrote: Sat Apr 06, 2024 10:36 Speaking of hypocrisy, most of these same people throwing out their opinions of what is not a mod are the same people for DECADES have saying AD NAUSEUM that Commander Keen is outdated:

---Dying in one hit, a health system should be used
---Loading and saving, checkpoints should be used
---extra lives are meaningless
---score is meaningless
---offers no meaningful rewards
--- most of all Keen needs to start using more game mechanics that modern video games use

I hate to have to be the one to tell you this, but these modern day game mechanics have been created for games played on modern game consoles, not on a mac or a PC. Most if not all of these mechanics are impossible for a DOS game on a PC. The same people that want modders to operate within superficial limitations of the original game engine and if they don't they are not mods, but in the same breath think that Keen should adapt the game mechanics of modern video games created by billion dollar companies, and somehow they will be mods. Yet a modder incorporating some of these game mechanics you demand into a mod, but use a source code to do it... NOPE! It's not a mod!!! Seriously?
I'm not sure who are these people you're talking about. The only people I've seen who support modern mechanics in Keen are also supportive of source modding, and some of them have even made source mods. I may not have been the most supportive of source modding, but I've always disagreed with people in that Keen was outdated (otherwise I would've added infinite lives and in-level score to my mods), although I sometimes worry that I might not able to enjoy the original Keen games as much without the QoL improvements that FITF added.
Quillax Ship - Where you can download all of my cool, keen mods and super-marvelous fangames!
Post Reply