Recreated Keen 4-6 Source Code

Here is where to post about the latest Commander Keen fangame or modification you've finished, a new website you've made, or another Keen-related creation.
User avatar
Pandakeen
Vortininja
Posts: 160
Joined: Tue Aug 03, 2021 9:54
Location: pckf,
Contact:

Re: Recreated Keen 4-6 Source Code

Post by Pandakeen »

Ah, it was set to BC20 and I'm only using BC31. Fixed that, but now it's showing this message:

Error: Segment _DATA exceeds 64K

:dead2
Calvero
Vortininja
Posts: 98
Joined: Tue Jan 29, 2008 15:31

Re: Recreated Keen 4-6 Source Code

Post by Calvero »

Just use BC20, BC30 and LZEXE, and create 100% identical copies of the original v1.4 executables before trying BC31.
User avatar
Quillax
Vorticon Elite
Posts: 554
Joined: Sat Mar 14, 2015 14:41
Location: Quillax Ship
Contact:

Re: Recreated Keen 4-6 Source Code

Post by Quillax »

gerstrong wrote:
Have you looked into this? I'm just wondering -- I'm sure it'll be very useful for making Commander Genius more accurate! :)
Quillax Ship - Where you can download all of my cool, keen mods and super-marvelous fangames!
gerstrong
Vorticon Elite
Posts: 1244
Joined: Wed Dec 31, 2008 14:44
Location: Frankfurt - Germany
Contact:

Re: Recreated Keen 4-6 Source Code

Post by gerstrong »

Quillax wrote: Wed Oct 20, 2021 3:20
gerstrong wrote:
Have you looked into this? I'm just wondering -- I'm sure it'll be very useful for making Commander Genius more accurate! :)
Not yet and it is nothing I have as high priority on my list. However if someone wants to integrate it, feel free to do so.

The challenge with the current engine is that it uses different framerates and a lot of home made code which brings other advantages, but gives the feeling of not "more accurate". Of course it is not like the original, but the original also lacks of multiplayer mode, higher framerates, different resolutions, better gamepad support to just name a few which we do provide.

So one thing that can be done, is to compare the code and check how to integrate it in terms of equivalent logic and see how possible it is to still keep the advantages that now exist.

Another thing that can be done is to create an entirely new engine within CG. Currently we have four:
  • One for the Commander Keen Galaxy games (the one you compared)
  • A Keen Dreams one (Based on the work of Refkeen)
  • The Vorticon engine (which obviously plays Keen1-3)
  • Cosmos Cosmic Adventure engine (got recently added)
There could exist another one that is Galaxy Keen but with highly accurate physics based on such code.

If someone wants to cooperate on that he is also welcome to do so and we would be happy to help on it. The source code is open, so everybody can fork and develop it further.

Since CG uses SDL2 and there exist lots of toolchains for building this on multiple system like Android, Windows, MacOS, Linux, Nintendo Switch, PS Vita, Raspberry Pi to just mention a few, any engine on CG will be spread to a lot of systems where these nice games make gamers beat their hearts faster. :love
Having fun developing stuff...
User avatar
Snortimer
Vortininja
Posts: 267
Joined: Wed Jul 22, 2009 8:04
Location: Toronto, Canada
Contact:

Re: Recreated Keen 4-6 Source Code

Post by Snortimer »

Amazing news, I think.

I'm very curious to take a look at it, especially after a guide is written up (if that's in the works). What's holding me back at the moment is that I've only ever coded anything in the web development languages (JavaScript, PHP, SQL, CSS etc.), which I learned within the past 2 years, so I should probably learn some basic C or C++ first.

An idea I had many years ago was to combine the gameplay of Commander Keen Galaxy (the platform levels and overhead maps) with Star Control II: The Ur Quan Masters (the travel through space between planets, dialogues with aliens and spaceship battles). Those two games seem to have the potential to complement each other without getting in each other's way. Now the source code for both is available (though one uses C++ and the other C), so it's now probably feasible in theory. (though it'll probably be a long time before I try it - still too "new" to this)
Member since at least 1998 with... ah... some long absences. I was even a moderator at one point. I'll probably keep coming back here and there as long as this place still exists.
mary4
Meep
Posts: 2
Joined: Fri Nov 18, 2022 0:13

Re: Recreated Keen 4-6 Source Code

Post by mary4 »

:dead2 i get "Error: Segment _DATA exceeds 64K" when trying to compile with borland C 3.1. the error is thrown when linking commander keen 4

i got it i had to run the command CK4PATCH.EXE ripck4.pat keen4e.exe to get some stuff then i had to run make with those ck4 files then i could compile! :D
Last edited by mary4 on Sat Nov 19, 2022 23:03, edited 2 times in total.
clb
Grunt
Posts: 15
Joined: Mon Sep 04, 2023 21:34

Re: Recreated Keen 4-6 Source Code

Post by clb »

I tried to compile Keen 4 1.4 and Keen 5 1.4 today, but oddly, I run into an issue that after pressing enter on the ASCII text intro screen, and when the Terminator screen should appear, the game screen stays blank and is effectively hung.

I am running on a 486 DX 80 MHz, with Borland Turbo C++ 3.0 installed from original disk box that I bought on eBay in 2020 (yeah, I did Borland Turbo C++ collecting :p).

Got Keen 4 1.4 from https://keenwiki.shikadi.net/wiki/Keen_4_Versions , Keen5 1.4 I had lying around. Then followed the readme.txt (copied exes into keen4-6\static and ran ripnmake.bat). That gave me this:

https://imgur.com/a/xR2v9S0

Despite the error messages, I think it worked? (maybe the errors related to attempting to process nonexisting KEEN6 files?) I did get various .OBJ files in the directories: https://imgur.com/a/PLY33g3 (that listing also contains post-build files after the steps below)

I then opened one of the .PRJ files with e.g. "TC RCK4.PRJ" in KEEN4-6 directory, updated the INCLUDE and LIB directories in the build, and compiled.

Everything build correctly and I got a KEEN4-6\KEEN4\OBJ\RCK4.EXE. But unfortunately still a black screen.

In the README I found a notice about non-volatile variable accesses getting optimized out in a spinloop, so I tried declaring the extern LastScan as volatile. Although README states that optimization issue should only have affected Borland Turbo C++ 3.1 and not 3.0.

Tried also compressing it with ..\..\..\LZEXE RCK4.EXE. (although I understand this should not be necessary to successfully run; just to make the binary smaller)

I did also try getting Borland Turbo C++ 2.0 from WinWorldPC (first try was without, just using Turbo C++ 3.0 and setting LIB dir to point there. Ran CLEANUP.BAT in between attempts)

Resulting tree looks like

https://imgur.com/a/DO4rGPR

Though unfortunately no success, just a black screen.

I did see a bug message in Quit() about VW_ClearVideo "bricking the system if screenseg is 0". Added a null check at beginning, but that did not affect the issue (well, there should not be EMS/XMS issues either). The original KEEN4E.EXE in the above directory does run ok.

Any help would be appreciated, I would be curious to play around with some timing code.

Btw, has there been any thought on pushing the Keen source code to GitHub, along with the needed toolchains? Maybe one branch could contain the "this branch builds the byte precise original output", and other branches could then evolve with fixes and ports, etc.
Last edited by clb on Tue Sep 05, 2023 15:54, edited 1 time in total.
clb
Grunt
Posts: 15
Joined: Mon Sep 04, 2023 21:34

Re: Recreated Keen 4-6 Source Code

Post by clb »

It looks like the build hangs in the function void DrawScan(Sint16 far *source, Uint8 far *dest) in CK_DEMO.C and never returns.

If I add an immediate return; to that function, and then run, I do avoid the very first immediate hang, but then get the following garbled output on screen:

https://imgur.com/a/ABKPFnl

that flickers for a bit, and then the game hangs again.

So that does suggest to me that some of the data is bad. Debugged the Null Pointer Assignments, but those were indeed just red herrings, as I don't have the CGA versions of the exe installed, nor KEEN6.

Not quite sure what to try next. If anyone has an idea, would be most welcome.
User avatar
Multimania
Vortininja
Posts: 84
Joined: Sat Nov 10, 2007 8:10
Location: Hiding in a small, cramped corner of the BwB megarocket.
Contact:

Re: Recreated Keen 4-6 Source Code

Post by Multimania »

clb wrote: Tue Sep 05, 2023 15:52 I am running on a 486 DX 80 MHz, with Borland Turbo C++ 3.0 installed from original disk box that I bought on eBay in 2020 (yeah, I did Borland Turbo C++ collecting :p).

<...>

I did also try getting Borland Turbo C++ 2.0 from WinWorldPC (first try was without, just using Turbo C++ 3.0 and setting LIB dir to point there. Ran CLEANUP.BAT in between attempts)
Can't check now, but IIRC, Keen was compiled with Borland C++, not Borland Turbo C++. I think the version here worked. Turbo C++ was used for Keens 1–3, I think, so I'm envious of your collection regardless. :-)
clb wrote: Tue Sep 05, 2023 15:52 Btw, has there been any thought on pushing the Keen source code to GitHub, along with the needed toolchains? Maybe one branch could contain the "this branch builds the byte precise original output", and other branches could then evolve with fixes and ports, etc.
I know Frenkel has his tandy ports on GitHub here, which also have a number of bugfixes. And Omnispeak has Win/Mac/Linux ports, as well as there being ports for N64, Amiga, 3DS, etc floating around. Still, I agree that it'd be good to have the unmodified keensource456 package uploaded somewhere for easy reference and linking. The Keen Dreams archive is a good example, too.
clb
Grunt
Posts: 15
Joined: Mon Sep 04, 2023 21:34

Re: Recreated Keen 4-6 Source Code

Post by clb »

Ohh, right. Yeah I can confirm that switching to build with Borland C++ 3.0 from WinWorldPC fixed the issue and I am now able to build. Sweet!
User avatar
Frenkel
Vortininja
Posts: 33
Joined: Sat Jan 01, 2022 11:25

Re: Recreated Keen 4-6 Source Code

Post by Frenkel »

There are a bunch of Keen related repositories at https://github.com/net-lisias-keen
keensource456 will fit right in.
clb
Grunt
Posts: 15
Joined: Mon Sep 04, 2023 21:34

Re: Recreated Keen 4-6 Source Code

Post by clb »

Thanks. I pushed my mod up to https://github.com/juj/KEEN70HZ for anyone interested. That was a fun little project. :dopefish
User avatar
K1n9_Duk3
Vorticon Elite
Posts: 781
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

Re: Recreated Keen 4-6 Source Code

Post by K1n9_Duk3 »

Before I start, please let me remind you that Turbo C++ is not the product you should be looking for if you want to create mods based on my reconstructed source code. Turbo C++ does not include Turbo Assembler (TASM), which is required for building an executable from this source code. You should really be looking for Borland C++ 3.0 or 3.1, which come bundled with a copy of TASM.

You can work with Turbo C++ 3.0 if that's the only version available to you and you already have a copy TASM anyway (and you modify the code as mentioned below), but I wouldn't recommend it.

- - -

I tried compiling my reconstructed Keen 4-6 code with Turbo C++ 3.0 earlier today. By the looks of it, Borland botched their implementation of huge pointer arithmetics in Turbo C++ 3.0 and that's what prevented the code from being able to decompress the graphics (and audio) data correctly and caused the game to lock up after the text-based startup screen.

When the code uses var++ or var-- on a huge pointer variable, Turbo C++ 3.0 ends up generating code that operates on the first 4 bytes of data pointed to by the huge pointer instead of adjusting the huge pointer variable itself. At least that is what happened in the version I tested. Compile and run the following code to see it yourself:

Code: Select all

#include <stdio.h>

#define TESTNUMBER 0xDEADC0DE
void main(void)
{
	long number = TESTNUMBER;
	long huge *pointer = &number;
	
	printf("number should stay constant, only pointer should change:\n"
	       "testing ++ and --\n");
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	pointer++;
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	pointer--;
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	
	if (pointer == &number)
		printf("pointer matches initial address\n");
	else
		printf("pointer doesn't match initial address\n");
	
	if (number == TESTNUMBER)
		printf("number still ok\n");
	else
		printf("number corrupted\n");
	
	number = TESTNUMBER;
	pointer = &number;
	
	printf("\ntesting += 1 and -= 1\n");
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	pointer += 1;
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	pointer -= 1;
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	
	if (pointer == &number)
		printf("pointer matches initial address\n");
	else
		printf("pointer doesn't match initial address\n");
	
	if (number == TESTNUMBER)
		printf("number still ok\n");
	else
		printf("number corrupted\n");
	
	number = TESTNUMBER;
	pointer = &number;
	
	printf("\ntesting indirect increment\n");
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	pointer = pointer + 1;
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	pointer = pointer - 1;
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	
	if (pointer == &number)
		printf("pointer matches initial address\n");
	else
		printf("pointer doesn't match initial address\n");
	
	if (number == TESTNUMBER)
		printf("number still ok\n");
	else
		printf("number corrupted\n");
}
I was able to get an executable that decompresses the audio and graphics data correctly by changing the code at the beginning of CAL_HuffExpand() from this:

Code: Select all

  source++;	// normalize
  source--;
  dest++;
  dest--;

  sourceseg = FP_SEG(source);
  sourceoff = FP_OFF(source);
  destseg = FP_SEG(dest);
  destoff = FP_OFF(dest);
  endoff = destoff+length;
to this:

Code: Select all

  // get normalized segment and offset for source and dest:
  sourceseg = FP_SEG(source) + FP_OFF(source) / 16;
  sourceoff = FP_OFF(source) & 15;
  destseg = FP_SEG(dest) + FP_OFF(dest) / 16;
  destoff = FP_OFF(dest) & 15;
  endoff = destoff+length;
After that change, the Terminator intro worked fine and I was able to play any level I've tried.

Please note that CAL_HuffExpand() is not the only routine that uses the botched huge pointer arithmetics. You will at least need to modify CA_RLEWCompress() as well to get a fully functional executable when using Turbo C++ 3.0 to compile the code. Simply replacing any "huge *" with "far *" in CA_RLEWCompress() should do the trick -- that routine is only used for compressing the maps stored in saved games and the maps cannot be larger than 64k anyway, so huge pointers are not necessary.
Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net
User avatar
Frenkel
Vortininja
Posts: 33
Joined: Sat Jan 01, 2022 11:25

Re: Recreated Keen 4-6 Source Code

Post by Frenkel »

clb wrote: Tue Sep 05, 2023 15:52 Btw, has there been any thought on pushing the Keen source code to GitHub
https://github.com/sparky4/keen4-6
Post Reply