Reflection Keen (ports of Keen Dreams and all 3D Catacombs)

Here is where to post about the latest Commander Keen fangame or modification you've finished, a new website you've made, or another Keen-related creation.
NY00123
Vorticon Elite
Posts: 442
Joined: Sun Sep 06, 2009 19:36

Reflection Keen (ports of Keen Dreams and all 3D Catacombs)

Post by NY00123 » Fri Sep 26, 2014 21:58

Last release on GitHub (Windows and Linux installers, Android packages and source code archive): https://github.com/NY00123/refkeen/rele ... e-20170401

Link to GitHub repository: https://github.com/NY00123/refkeen/

NOTE: For anybody looking for an archive to manually extract:
- Windows: 7-Zip (http://www.7-zip.org/) can be used to extract the contents of the Windows installer (NSIS-based) in which you're (probably) interested.
- Linux: In a similar manner, each of the Linux installers (MojoSetup-based) can be extracted as a ZIP file, although you may need to first rename the downloaded installer file so it ends with .zip. You want the contents of the "data" directory.

ANOTHER NOTE: If you've downloaded one of the Linux installers, you may need to set the file to be executable before you can actually run the installer itself.



UPDATE (Oct 13 2016): Following the last release, download links have been updated again. Links to Android packages have been introduced, and some minor grouping has been done. It's possible there won't be another comment of update in this post for any following release (except for updating the links, of course).

UPDATE (Nov 20 2015): Following the last release, let's relocate the download links to the beginning of this post. Otherwise, though, this post is not touched.

UPDATE (Mar 13 2015): It's a little bit overdue, but finally there is a title for the whole codebase now, not just specific source ports. It shouldn't be a surprise, but this name is Reflection Keen. This comes from a release which, along with some other modifications, has experimental "modern" support for game controllers (with the layout of the Xbox 360 controller or similar).

UPDATE (Dec 20 2014): Support for the whole Catacomb Adventure Series is now in. This release should be compatible with the 3D Catacomb titles currently available from GOG.com, or to be more specific: Catacomb 3-D: The Descent v1.22, The Catacomb Abyss v1.24, The Catacomb Armageddon v1.02 and The Catacomb Apocalypse v1.01.

Please note that Armageddon/Apocalypse saved games are generally incompatible with the DOS executables. For more details you can check out the README file, although admittedly a few details are a bit technical, but there's a workaround that may partially work.

ImageImage

UPDATE (Nov 30 2014): Support for Catacomb 3-D (The Descent) has been added. It has taken much less time to add this than Abyss, given that Abyss is close to a direct derivative of 3-D with some added "Gamer's Edge" specific code. As expected, this port is called "Ref Catacomb 3-D".

Image

UPDATE (Oct 25 2014): Support for Catacomb Abyss has been added. One main reason is that both Keen Dreams and Catacomb Abyss share a lot of common code.

As a consequence, and just to be a bit original, the Keen Dreams port has been renamed Ref Keen Dreams. Here, "Ref" stands for "Reflection". Similarly there should be a "Ref Catacomb Abyss" port now. These ports are still inspired by Chocolate Doom, though.

Image

Earlier updates and post contents aren't modified for now.

UPDATE (Oct 11 2014): Support for a couple of more versions (with the same EGA graphics) has been added, along with other misc. changes and fixes. Furthermore, the two screenshots in this post have been replaced with smaller ones (hopefully filling lost of this topic's first page), also taken from the release of Oct 11 2014 so you should see the (basically useless) overscan borders.

UPDATE (Oct 09 2014): EGA graphics emulation has been added, and this port should support the shareware release of Keen Dreams, version 1.13.

GitHub users: Watch out as all work has moved off the cga branch to master, and all branches differing from master have been removed.

A screenshot is shown for illustration:

Image

UPDATE (Sep 29/30 2014): This post shall be edited with up-to-date download links for every new release.

Original post contents (with a few edits):

Hey all,

Now that we have original source codes for Keen Dreams, what about a port? Well, since CGA graphics handling is somewhat less complicated than the EGA counterpart, I think, and I already kind-of started the work even before the release (think, well, little bits of Catacomb 3D)...

Image

As expected, this port aims to reproduce vanilla Keen Dreams bugs, at least if it isn't too difficult (e.g., anything that highly depends on the memory layout).

Note that this requires a copy of Keen Dreams CGA version 1.05.

If you have any question, maybe it's answered in the README file bundled with any of the archives below. But if you still want EGA graphics, well, as of now (apart from the DOS executables) I'm aware of this progress by Multimania/sulix/David Gow, who has also given me help beforehand (and also an important hint for this port): https://github.com/sulix/keen-dreams-sdl2
Last edited by NY00123 on Sat Apr 01, 2017 9:49, edited 23 times in total.

User avatar
Lunick
Vortininja
Posts: 267
Joined: Tue Jun 01, 2010 10:37
Location: Melbourne, Australia
Contact:

Post by Lunick » Sat Sep 27, 2014 6:57

Great work ny00123 :birthday
Lunick is Here

User avatar
Levellass
S-Triazine
Posts: 5261
Joined: Tue Sep 23, 2008 6:40

Post by Levellass » Sun Sep 28, 2014 7:27

CGA handling is simpler? I know it doesn't do the whole 4 planes thing, but it always seemed rather tricky to me.

Has anyone added the titlescreen format to the modding wiki? I assume this can decompress it.
What you really need, not what you think you ought to want.

NY00123
Vorticon Elite
Posts: 442
Joined: Sun Sep 06, 2009 19:36

Post by NY00123 » Mon Sep 29, 2014 22:21

Hi all, and thanks for the comments so far!

Now, a little update should be available for download (first post has been edited with updated download links). Guess a few main points I can mention about the release are (partial) saved game compatibility and kind-of implementations of a few feature requests (a bit about mouse cursor behaviors, and manual window resolution setting).

Changelog

Sep 29 2014 (v0.8.1):
- Compatibility with saved games from vanilla Keen Dreams CGA v1.05 is in. As expected, it can still be buggy (in fact, vanilla Keen also has its limits/bugs with this).
- As a possibly indirect consequence, a vanilla Keen bug in StateMachine/DoActor leading to a crash in Chocolate Keen Dreams has been found (NULL pointer dereference). Workaround is applied now.
- The environment variables from last release are gone. Instead, you should have the configuration file chocolate-keen-dreams.cfg to fiddle with (created on launch if it doesn't exist).
- Custom fullscreen and windowed resolutions can be set in the configuration file. Furthermore, aspect correction can be toggled off (e.g., in case screen burn-in is a concern).
- Mouse cursor lock can be toggled off (from the cfg). By default the cursor is now unlocked in a non-fullscreen window. Chances are the behaviors may still feel a bit off, but at least we have that.
- Loading window with bars should display (while loading a map), although it's shown for about a short moment on sufficiently fast machines.
Levellass wrote:CGA handling is simpler? I know it doesn't do the whole 4 planes thing, but it always seemed rather tricky to me.
Well, it is about the 4 planes handling, indeed. I've justed wanted to have less troubles with the porting, considering the fact that any little mistake can mess up the whole thing.

As a few would expect, the CGA graphics have one byte per 4 pixels in a straightforward linear format. Keen Dreams for the CGA generally draws these graphics to system memory (which differs from the case with the EGA), and a one-time update function copies the contents to CGA memory. The trickiest part here is the porting of ASM code, especially while taking 16-bit offset wrapping into account everywhere (Multimania/sulix/David Gow has spotted similar wrappings earlier with EGA graphics handling, while working on his port of Keen Dreams). Truly, the graphics should be stored in an interlaced pattern within CGA memory, but that's relatively insignificant.
Has anyone added the titlescreen format to the modding wiki? I assume this can decompress it.
Can't say I know much about it, but it looks like a map is loaded before the titlescreen is shown (map 20).

NY00123
Vorticon Elite
Posts: 442
Joined: Sun Sep 06, 2009 19:36

Post by NY00123 » Fri Oct 03, 2014 10:08

Alright, to begin with, you can all ignore my last comment to Levellass about titlescreen decompression. Looks like it doesn't apply to all versions of Keen Dreams (CGA v1.05 is a good (counter)example).

Next, I know there is a general preference for EGA graphics, but I'm still not there at the moment. Nevertheless, another minor update is up, which can improve the graphical output so scaling artifacts are less noticeable, yet the output still doesn't look too blurry. Furthermore, in case of a crash/hang, you may have some luck by changing the sound sample rate to something different from 49716Hz (the OPL rate).

Changelog

Oct 3, 2014 (v0.8.2):
- Bilinear filtering can now be toggled on if hardware acceleration is in use.
- In case off-screen rendering is supported, it can be used for two-step
scaling of the graphical output: First nearest-neighbor interpolation with
the added "scalefactor" setting, and then bilinear.
- Sync to VBlank can be manually toggled now (if supported on the used setup).
By default "vsync=auto" is used, currently implying "off" (with CGA graphics).
- Live sound interpolation has been implemented, and it's possible to specify
a different sound sample rate, while the rate used for OPL emulation is still
49716Hz. Currently 49716 is the default value
- Fixed a bug: One cfg setting or more not written to file when it's expected.
- As of this version, the cfg file is (re)written on launch. This is done so
new settings can appear in the file if it's not up-to-date.

User avatar
Levellass
S-Triazine
Posts: 5261
Joined: Tue Sep 23, 2008 6:40

Post by Levellass » Thu Oct 09, 2014 7:56

NY00123 wrote:Alright, to begin with, you can all ignore my last comment to Levellass about titlescreen decompression. Looks like it doesn't apply to all versions of Keen Dreams (CGA v1.05 is a good (counter)example).

Wait... what comment? Was I given information? Where?!
What you really need, not what you think you ought to want.

NY00123
Vorticon Elite
Posts: 442
Joined: Sun Sep 06, 2009 19:36

Post by NY00123 » Thu Oct 09, 2014 14:28

OK, this is what at least a few of you may have wanted. Basically, EGA graphics drawing is in and there's support for the Shareware release of Keen Dreams, version 1.13. This includes possibly-buggy compatibility with saved games.

Changelog

Oct 9, 2014 (v0.8.8):
- EGA graphics emulation has been added, and Chocolate Keen Dreams should be
compatible with the Shareware release of Keen Dreams, version 1.13.
- This includes compatibility with saved games (each Chocolate Keen exe should
be compatible with saved games of the corresponding release for DOS only).
- Two separate executables are used for the CGA and Shareware releases.
- chocolate-keen-dreams-cga.exe has been renamed to the shorter filename of
chocolate-kdreams-cga.exe. There's also chocolate-kdreams-shar.exe now.
- Difference from behaviors of vanilla Keen: For the Shareware release,
you don't need to select a "START" executable in order to launch the game.
But you can still get the message telling you need to type START by adding the
command-line argument of /detour (exact opposite of original behaviors).
- High scores table glitch is fixed.
- Other miscellaneous fixes.
Levellass wrote:
NY00123 wrote:Alright, to begin with, you can all ignore my last comment to Levellass about titlescreen decompression. Looks like it doesn't apply to all versions of Keen Dreams (CGA v1.05 is a good (counter)example).

Wait... what comment? Was I given information? Where?!
Not really, it's just a reply to a misleading information of mine which is relevant to older versions loading the titlescreen off the tileset:
NY00123 wrote:
Levellass wrote:Has anyone added the titlescreen format to the modding wiki? I assume this can decompress it.
Can't say I know much about it, but it looks like a map is loaded before the titlescreen is shown (map 20).

User avatar
Levellass
S-Triazine
Posts: 5261
Joined: Tue Sep 23, 2008 6:40

Post by Levellass » Fri Oct 10, 2014 11:54

Yeah, in some Keen Dreams a map is used for the tile, or at least an attempt is made to do this (Hence the titlescreen tiles in the tileset.) but in some versions at least a separate titlescreen file is used. It's a bitmap in the form of that weird Apple II thingy which can be compressed in two different ways. One is simple RLE and the other appears to be either LZW, Huffman or LZH.
What you really need, not what you think you ought to want.

NY00123
Vorticon Elite
Posts: 442
Joined: Sun Sep 06, 2009 19:36

Post by NY00123 » Sat Oct 11, 2014 20:40

Well, I guess there aren't a lot of "interesting" updates to see anymore (like the introduction of EGA graphics support). But there's another release now, adding compatibility with a couple of more versions and having a few other changes, including bug fixes. One of these changes is that with the last version, black overscan borders are permanently drawn. Don't ask me why. (ok, there is some memory management related code which blinks the borders, but I'm not sure it is ever executed in practice. Furthermore, maybe this will become useful for somebody later.)

Changelog

Oct 11, 2014 (v0.8.10):
- Fixed finale text printing (more generally, modified some functions
to better handle original code leading to undefined behaviors per the
C standard, or at least to crashes).
- Fixed King Boobus Tuber explosion.
- Added compatibility with the registered release, v1.93, and the Shareware
release, v1.20. Taking a look at the source codes release, they are both
almost identical to v1.13.
- Overscan borders are drawn now, even if permanently colored black (MM_SortMem
flashes the borders, but chances are the function is never called in practice).
- More miscellaneous fixes.
Levellass wrote:Yeah, in some Keen Dreams a map is used for the tile, or at least an attempt is made to do this (Hence the titlescreen tiles in the tileset.) but in some versions at least a separate titlescreen file is used. It's a bitmap in the form of that weird Apple II thingy which can be compressed in two different ways. One is simple RLE and the other appears to be either LZW, Huffman or LZH.
I have this feeling there were plans to use this kind of CMP file as a general container for all data, given that it can theoretically have multiple internal "files". Maybe this was done for some other Softdisk title; I don't know at the moment.

As of Dreams, looks like the compression of the title screen was done with LZH.

User avatar
Levellass
S-Triazine
Posts: 5261
Joined: Tue Sep 23, 2008 6:40

Post by Levellass » Sun Oct 12, 2014 6:31

NY00123 wrote:I have this feeling there were plans to use this kind of CMP file as a general container for all data, given that it can theoretically have multiple internal "files". Maybe this was done for some other Softdisk title; I don't know at the moment.

As of Dreams, looks like the compression of the title screen was done with LZH.
You'll want to check out Dave 3 and 4, using the same engine a lot of their files were contained in such a way.
What you really need, not what you think you ought to want.

NY00123
Vorticon Elite
Posts: 442
Joined: Sun Sep 06, 2009 19:36

Post by NY00123 » Sat Oct 25, 2014 12:46

It is time for another update, probably the most significant since v0.8.8. I'm quoting the very first post of this topic after editing it, with possible modifications.

Support for Catacomb Abyss has been added. One main reason is that both Keen Dreams and Catacomb Abyss share a lot of common code. Most source code files are separate, though, and so is the EXE.

As a consequence, and just to be a bit original, the Keen Dreams port has been renamed Ref Keen Dreams. Here, "Ref" stands for "Reflection". Similarly there should be a "Ref Catacomb Abyss" port now. These ports are still inspired by Chocolate Doom, though.

Ref Catacomb Abyss is based on the source code release from June 2014, modified to be compatible with the Shareware release, v1.13 (QA [0]).

Download links should be found in the very first post of this topic. Please be warned that the configuration file chocolate-keen-dreams.cfg is now renamed refkdreams.cfg. Similarly, Ref Catacomb Abyss uses refcatabyss.cfg.

Changelog

Oct 25, 2014 (v0.9.0):
- RENAMING OF PORT: The Keen Dreams port has been renamed "Ref Keen Dreams".
This is done to be a just a little bit more original, and also follows the
addition of a new source port called "Ref Catacomb Abyss" (see next point).
Note that the source codes still have internal mentions of "CHOCO" or similar,
but from the user's point of view "Ref" is the new prefix for the two ports.
- Support for Catacomb Abyss (which shares a lot of code with Keen Dreams)
has been added. It is based on the original source code release from June 2014,
while modified to be compatible with the data from the Shareware release,
v1.13 (QA [0]).
- Apart from major modifications like ports of 3D scaling routines, there are
also some minor changes, like the support of a 640x200 graphics mode emulation
(for Catacomb Abyss help section, internally similar to 320x200) and overscan
borders flashing (technically supported since v0.8.10, but actually used now).
- There should be compatibility with saved games of the exact same Catacomb
Abyss release, but this can be buggy as usual. Furthermore, the way saved games
names are displayed in the corresponding dialogs (via the F3 or F4 key) may
seem a bit weird. This can be improved, but for now we have that.
- Another related complication is the fact that the Catacomb Abyss port is
compatible with a platform where case-sensitive filesystems are commonly used,
while under DOS this is not the case and filenames tend to be renamed uppercase
automatically. As of this version of the Catacomb Abyss port, newly saved
games' filenames are automatically converted to uppercase. The same applies
when typing a name for game loading (even if lowercase filenames are shown).
This shouldn't be a problem on case-insensitive filesystems, even if they're
case-preserving.
- Partial compatibility breakage: chocolate-keen-dreams.cfg is renamed
refkdreams.cfg. Similarly refcatabyss.cfg is used for Catacomb Abyss.

User avatar
Levellass
S-Triazine
Posts: 5261
Joined: Tue Sep 23, 2008 6:40

Post by Levellass » Tue Oct 28, 2014 6:03

So then... Catacomb Abyss is a freeware game? If so, where can I get it? Might be nice to code up a graphics editor for it.
What you really need, not what you think you ought to want.

User avatar
Lunick
Vortininja
Posts: 267
Joined: Tue Jun 01, 2010 10:37
Location: Melbourne, Australia
Contact:

Post by Lunick » Tue Oct 28, 2014 7:27

I think there are two links to download it on the release page of the port but, here is a safe link here http://www.classicdosgames.com/game/The ... Abyss.html Download 1.13
Lunick is Here

User avatar
Levellass
S-Triazine
Posts: 5261
Joined: Tue Sep 23, 2008 6:40

Post by Levellass » Wed Oct 29, 2014 8:11

Well darn, this has wretchedly obtuse file storage. Really would like a nice clear explanation of the compression used.

Graphics came out with keengraph easy enough. (Got to LOVE robust code! I mean I just renamed the file extensions to CK4!)

Don't know if anyone cares but here's all the extracted graphics: https://dl.dropboxusercontent.com/u/3940020/ABYSSKG.zip
What you really need, not what you think you ought to want.

NY00123
Vorticon Elite
Posts: 442
Joined: Sun Sep 06, 2009 19:36

Post by NY00123 » Sun Nov 30, 2014 23:04

I should've really replied to (at least some of) the preceding posts beforehand. Guess I can do it in this post. I'm announcing another update first, though. Again it's based on the just-edited initial post of mine.

Support for Catacomb 3-D (The Descent) has been added. It has taken much less time to add this than Abyss, given that Abyss is close to a direct derivative of 3-D with some added "Gamer's Edge" specific code. As expected, this port is called "Ref Catacomb 3-D".

This release should be compatible with versions 1.00 and 1.22 of Catacomb 3-D.

Download links should be found in the very first post of this topic.

Changelog

Nov 30, 2014 (v0.9.6):
- Support for Catacomb 3-D (The Descent) is now in. This includes support for
versions 1.00 and 1.22, as well as saved game compatibility with each of these
separately (although this can be buggy as usual). A lot of ID Engine code is
shared between the two, while an earlier revision of the ID Engine used by
Keen Dreams remains separate.
- Intro and exit screens are added for Catacomb Abyss. The intro can be skipped
by adding the /skipintro command line argument (same as launching vanilla
CATABYSS.EXE Shareware v1.13 with a seemingly-random pattern as an argument).
- Known non-vanilla bug/limitation: Skull 'n' Bones is a bit slower than
vanilla if VSync is toggled on (given a refresh rate of 60Hz), so it's disabled
by default in Catacomb 3-D for now. For the sake of consistency it's also
disabled in Catacomb Abyss. It is enabled in Keen Dreams by default if
EGA graphics are used, though.
- The Catacomb Abyss Vanilla bug reproduction: Numbers not shown in HUD on
startup immediately.
- A few more misc. changes.
Levellass wrote:So then... Catacomb Abyss is a freeware game? If so, where can I get it? Might be nice to code up a graphics editor for it.
It's admittedly confusing, but while the original Catacomb title was released as freeware (along with Dangerous Dave as a part of the Gamer's Edge Sampler), that Catacomb Abyss release is tagged "Shareware". In practice, the Gamer's Edge Sampler had kind-of similar goals to these of the Shareware model, so the differences aren't that great, indeed.

I guess Lunick has given an answer regarding an available download.
Levellass wrote:Well darn, this has wretchedly obtuse file storage. Really would like a nice clear explanation of the compression used.

Graphics came out with keengraph easy enough. (Got to LOVE robust code! I mean I just renamed the file extensions to CK4!)

Don't know if anyone cares but here's all the extracted graphics: https://dl.dropboxusercontent.com/u/3940020/ABYSSKG.zip
Nice to see this! It looks like there are a few unused graphics originally coming from Catacomb 3-D at the least (yeah, 4TIL-8-UNMASKED, bits of control panel gfx).

On a related note, while totally unused, the released Catacomb Abyss source code has the file ID_US_2.C with the Keen 4-6/Cat3D control panel code, although the code for Skull 'n' Bones (Paddle War) is commented out. Maybe there were plans to keep the control panel code in, just without this mini-game.

Eventually, though, it was decided that it won't even be compiled. On the other hand, as I'm sure you've seen, there's a lot of new "Gamer's Edge library" code added (more than used in all releases of Keen Dreams). There's also a a new intro sequence, from which you can pick the difficulty (either Novice or Warrior). For some reason it resides in a separate INTRO.EXE file. I've recently had the thought that maybe it's a way to save memory.

Finally, about this new kind of file storage (these ABS files), I think this is never used in the game itself, just in INTRO.EXE and LOADSCN.EXE. The way these shapes are loaded is very similar to what's used for the compressed title screen in some of the Keen Dreams releases, even the same kind of LZH compression, although there may be some differences (e.g., not using ID_MM's handlers for memory management, just plain farmalloc/farfree).

Post Reply