Page 4 of 4

Re: Recreated Keen 4-6 Source Code

Posted: Tue Oct 12, 2021 11:27
by Pandakeen
Ah, it was set to BC20 and I'm only using BC31. Fixed that, but now it's showing this message:

Error: Segment _DATA exceeds 64K

:dead2

Re: Recreated Keen 4-6 Source Code

Posted: Sun Oct 17, 2021 15:30
by Calvero
Just use BC20, BC30 and LZEXE, and create 100% identical copies of the original v1.4 executables before trying BC31.

Re: Recreated Keen 4-6 Source Code

Posted: Wed Oct 20, 2021 3:20
by Quillax
gerstrong wrote:
Have you looked into this? I'm just wondering -- I'm sure it'll be very useful for making Commander Genius more accurate! :)

Re: Recreated Keen 4-6 Source Code

Posted: Fri Nov 05, 2021 6:13
by gerstrong
Quillax wrote: Wed Oct 20, 2021 3:20
gerstrong wrote:
Have you looked into this? I'm just wondering -- I'm sure it'll be very useful for making Commander Genius more accurate! :)
Not yet and it is nothing I have as high priority on my list. However if someone wants to integrate it, feel free to do so.

The challenge with the current engine is that it uses different framerates and a lot of home made code which brings other advantages, but gives the feeling of not "more accurate". Of course it is not like the original, but the original also lacks of multiplayer mode, higher framerates, different resolutions, better gamepad support to just name a few which we do provide.

So one thing that can be done, is to compare the code and check how to integrate it in terms of equivalent logic and see how possible it is to still keep the advantages that now exist.

Another thing that can be done is to create an entirely new engine within CG. Currently we have four:
  • One for the Commander Keen Galaxy games (the one you compared)
  • A Keen Dreams one (Based on the work of Refkeen)
  • The Vorticon engine (which obviously plays Keen1-3)
  • Cosmos Cosmic Adventure engine (got recently added)
There could exist another one that is Galaxy Keen but with highly accurate physics based on such code.

If someone wants to cooperate on that he is also welcome to do so and we would be happy to help on it. The source code is open, so everybody can fork and develop it further.

Since CG uses SDL2 and there exist lots of toolchains for building this on multiple system like Android, Windows, MacOS, Linux, Nintendo Switch, PS Vita, Raspberry Pi to just mention a few, any engine on CG will be spread to a lot of systems where these nice games make gamers beat their hearts faster. :love

Re: Recreated Keen 4-6 Source Code

Posted: Wed Nov 17, 2021 11:29
by Snortimer
Amazing news, I think.

I'm very curious to take a look at it, especially after a guide is written up (if that's in the works). What's holding me back at the moment is that I've only ever coded anything in the web development languages (JavaScript, PHP, SQL, CSS etc.), which I learned within the past 2 years, so I should probably learn some basic C or C++ first.

An idea I had many years ago was to combine the gameplay of Commander Keen Galaxy (the platform levels and overhead maps) with Star Control II: The Ur Quan Masters (the travel through space between planets, dialogues with aliens and spaceship battles). Those two games seem to have the potential to complement each other without getting in each other's way. Now the source code for both is available (though one uses C++ and the other C), so it's now probably feasible in theory. (though it'll probably be a long time before I try it - still too "new" to this)

Re: Recreated Keen 4-6 Source Code

Posted: Fri Nov 18, 2022 0:38
by mary4
:dead2 i get "Error: Segment _DATA exceeds 64K" when trying to compile with borland C 3.1. the error is thrown when linking commander keen 4

i got it i had to run the command CK4PATCH.EXE ripck4.pat keen4e.exe to get some stuff then i had to run make with those ck4 files then i could compile! :D

Re: Recreated Keen 4-6 Source Code

Posted: Tue Sep 05, 2023 15:52
by clb
I tried to compile Keen 4 1.4 and Keen 5 1.4 today, but oddly, I run into an issue that after pressing enter on the ASCII text intro screen, and when the Terminator screen should appear, the game screen stays blank and is effectively hung.

I am running on a 486 DX 80 MHz, with Borland Turbo C++ 3.0 installed from original disk box that I bought on eBay in 2020 (yeah, I did Borland Turbo C++ collecting :p).

Got Keen 4 1.4 from https://keenwiki.shikadi.net/wiki/Keen_4_Versions , Keen5 1.4 I had lying around. Then followed the readme.txt (copied exes into keen4-6\static and ran ripnmake.bat). That gave me this:

https://imgur.com/a/xR2v9S0

Despite the error messages, I think it worked? (maybe the errors related to attempting to process nonexisting KEEN6 files?) I did get various .OBJ files in the directories: https://imgur.com/a/PLY33g3 (that listing also contains post-build files after the steps below)

I then opened one of the .PRJ files with e.g. "TC RCK4.PRJ" in KEEN4-6 directory, updated the INCLUDE and LIB directories in the build, and compiled.

Everything build correctly and I got a KEEN4-6\KEEN4\OBJ\RCK4.EXE. But unfortunately still a black screen.

In the README I found a notice about non-volatile variable accesses getting optimized out in a spinloop, so I tried declaring the extern LastScan as volatile. Although README states that optimization issue should only have affected Borland Turbo C++ 3.1 and not 3.0.

Tried also compressing it with ..\..\..\LZEXE RCK4.EXE. (although I understand this should not be necessary to successfully run; just to make the binary smaller)

I did also try getting Borland Turbo C++ 2.0 from WinWorldPC (first try was without, just using Turbo C++ 3.0 and setting LIB dir to point there. Ran CLEANUP.BAT in between attempts)

Resulting tree looks like

https://imgur.com/a/DO4rGPR

Though unfortunately no success, just a black screen.

I did see a bug message in Quit() about VW_ClearVideo "bricking the system if screenseg is 0". Added a null check at beginning, but that did not affect the issue (well, there should not be EMS/XMS issues either). The original KEEN4E.EXE in the above directory does run ok.

Any help would be appreciated, I would be curious to play around with some timing code.

Btw, has there been any thought on pushing the Keen source code to GitHub, along with the needed toolchains? Maybe one branch could contain the "this branch builds the byte precise original output", and other branches could then evolve with fixes and ports, etc.

Re: Recreated Keen 4-6 Source Code

Posted: Tue Sep 05, 2023 18:12
by clb
It looks like the build hangs in the function void DrawScan(Sint16 far *source, Uint8 far *dest) in CK_DEMO.C and never returns.

If I add an immediate return; to that function, and then run, I do avoid the very first immediate hang, but then get the following garbled output on screen:

https://imgur.com/a/ABKPFnl

that flickers for a bit, and then the game hangs again.

So that does suggest to me that some of the data is bad. Debugged the Null Pointer Assignments, but those were indeed just red herrings, as I don't have the CGA versions of the exe installed, nor KEEN6.

Not quite sure what to try next. If anyone has an idea, would be most welcome.

Re: Recreated Keen 4-6 Source Code

Posted: Tue Sep 05, 2023 23:25
by Multimania
clb wrote: Tue Sep 05, 2023 15:52 I am running on a 486 DX 80 MHz, with Borland Turbo C++ 3.0 installed from original disk box that I bought on eBay in 2020 (yeah, I did Borland Turbo C++ collecting :p).

<...>

I did also try getting Borland Turbo C++ 2.0 from WinWorldPC (first try was without, just using Turbo C++ 3.0 and setting LIB dir to point there. Ran CLEANUP.BAT in between attempts)
Can't check now, but IIRC, Keen was compiled with Borland C++, not Borland Turbo C++. I think the version here worked. Turbo C++ was used for Keens 1–3, I think, so I'm envious of your collection regardless. :-)
clb wrote: Tue Sep 05, 2023 15:52 Btw, has there been any thought on pushing the Keen source code to GitHub, along with the needed toolchains? Maybe one branch could contain the "this branch builds the byte precise original output", and other branches could then evolve with fixes and ports, etc.
I know Frenkel has his tandy ports on GitHub here, which also have a number of bugfixes. And Omnispeak has Win/Mac/Linux ports, as well as there being ports for N64, Amiga, 3DS, etc floating around. Still, I agree that it'd be good to have the unmodified keensource456 package uploaded somewhere for easy reference and linking. The Keen Dreams archive is a good example, too.

Re: Recreated Keen 4-6 Source Code

Posted: Wed Sep 06, 2023 10:44
by clb
Ohh, right. Yeah I can confirm that switching to build with Borland C++ 3.0 from WinWorldPC fixed the issue and I am now able to build. Sweet!

Re: Recreated Keen 4-6 Source Code

Posted: Wed Sep 06, 2023 14:56
by Frenkel
There are a bunch of Keen related repositories at https://github.com/net-lisias-keen
keensource456 will fit right in.

Re: Recreated Keen 4-6 Source Code

Posted: Sat Sep 09, 2023 21:41
by clb
Thanks. I pushed my mod up to https://github.com/juj/KEEN70HZ for anyone interested. That was a fun little project. :dopefish

Re: Recreated Keen 4-6 Source Code

Posted: Wed Sep 13, 2023 15:51
by K1n9_Duk3
Before I start, please let me remind you that Turbo C++ is not the product you should be looking for if you want to create mods based on my reconstructed source code. Turbo C++ does not include Turbo Assembler (TASM), which is required for building an executable from this source code. You should really be looking for Borland C++ 3.0 or 3.1, which come bundled with a copy of TASM.

You can work with Turbo C++ 3.0 if that's the only version available to you and you already have a copy TASM anyway (and you modify the code as mentioned below), but I wouldn't recommend it.

- - -

I tried compiling my reconstructed Keen 4-6 code with Turbo C++ 3.0 earlier today. By the looks of it, Borland botched their implementation of huge pointer arithmetics in Turbo C++ 3.0 and that's what prevented the code from being able to decompress the graphics (and audio) data correctly and caused the game to lock up after the text-based startup screen.

When the code uses var++ or var-- on a huge pointer variable, Turbo C++ 3.0 ends up generating code that operates on the first 4 bytes of data pointed to by the huge pointer instead of adjusting the huge pointer variable itself. At least that is what happened in the version I tested. Compile and run the following code to see it yourself:

Code: Select all

#include <stdio.h>

#define TESTNUMBER 0xDEADC0DE
void main(void)
{
	long number = TESTNUMBER;
	long huge *pointer = &number;
	
	printf("number should stay constant, only pointer should change:\n"
	       "testing ++ and --\n");
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	pointer++;
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	pointer--;
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	
	if (pointer == &number)
		printf("pointer matches initial address\n");
	else
		printf("pointer doesn't match initial address\n");
	
	if (number == TESTNUMBER)
		printf("number still ok\n");
	else
		printf("number corrupted\n");
	
	number = TESTNUMBER;
	pointer = &number;
	
	printf("\ntesting += 1 and -= 1\n");
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	pointer += 1;
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	pointer -= 1;
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	
	if (pointer == &number)
		printf("pointer matches initial address\n");
	else
		printf("pointer doesn't match initial address\n");
	
	if (number == TESTNUMBER)
		printf("number still ok\n");
	else
		printf("number corrupted\n");
	
	number = TESTNUMBER;
	pointer = &number;
	
	printf("\ntesting indirect increment\n");
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	pointer = pointer + 1;
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	pointer = pointer - 1;
	printf(" number=%08lX -- pointer=%lX\n", number, pointer);
	
	if (pointer == &number)
		printf("pointer matches initial address\n");
	else
		printf("pointer doesn't match initial address\n");
	
	if (number == TESTNUMBER)
		printf("number still ok\n");
	else
		printf("number corrupted\n");
}
I was able to get an executable that decompresses the audio and graphics data correctly by changing the code at the beginning of CAL_HuffExpand() from this:

Code: Select all

  source++;	// normalize
  source--;
  dest++;
  dest--;

  sourceseg = FP_SEG(source);
  sourceoff = FP_OFF(source);
  destseg = FP_SEG(dest);
  destoff = FP_OFF(dest);
  endoff = destoff+length;
to this:

Code: Select all

  // get normalized segment and offset for source and dest:
  sourceseg = FP_SEG(source) + FP_OFF(source) / 16;
  sourceoff = FP_OFF(source) & 15;
  destseg = FP_SEG(dest) + FP_OFF(dest) / 16;
  destoff = FP_OFF(dest) & 15;
  endoff = destoff+length;
After that change, the Terminator intro worked fine and I was able to play any level I've tried.

Please note that CAL_HuffExpand() is not the only routine that uses the botched huge pointer arithmetics. You will at least need to modify CA_RLEWCompress() as well to get a fully functional executable when using Turbo C++ 3.0 to compile the code. Simply replacing any "huge *" with "far *" in CA_RLEWCompress() should do the trick -- that routine is only used for compressing the maps stored in saved games and the maps cannot be larger than 64k anyway, so huge pointers are not necessary.

Re: Recreated Keen 4-6 Source Code

Posted: Mon Oct 09, 2023 21:03
by Frenkel
clb wrote: Tue Sep 05, 2023 15:52 Btw, has there been any thought on pushing the Keen source code to GitHub
https://github.com/sparky4/keen4-6