Reverse Engineering Keen

Here, you can get help with anything or just chat about the original Commander Keen games.
Stickmanofdoom
Grunt
Posts: 3
Joined: Fri Nov 16, 2007 3:57

Reverse Engineering Keen

Post by Stickmanofdoom »

I was reading a slashdot article the other day, and happened across a comment about Commander Keen. But what really caught my attention, was that the comment was about reverse engineering the code, and it even provided the code. Check it out here: http://slashdot.org/comments.pl?sid=361915&cid=21372451 .
User avatar
entropicdecay
Mad Mushroom
Posts: 576
Joined: Thu Nov 01, 2007 15:00
Contact:

Post by entropicdecay »

Whoa. If this source code works, it's quite an amazing find. Quick, somebody compile it!

I never would've thought somebody would've already reverse engineered the source code for Commander Keen...

It looks pretty complicated... I know some programming and I've used a bit of C++ I think, but I don't understand this. Some of it looks more like assembly, maybe?
User avatar
ckguy
Bipship Engineer
Posts: 1169
Joined: Thu Nov 01, 2007 17:56
Location: Wakefield, RI, US
Contact:

Post by ckguy »

I'm going to hold off on being real excited for this until someone compiles it. And also, yeah it does look like a lot of assembly, written in a C format.
Galaxieretter
Arachnut
Posts: 891
Joined: Thu Nov 01, 2007 2:35
Location: Lancaster PA
Contact:

Post by Galaxieretter »

What I want to know is how does someone decompile a binary into a language? More specifically... how do *I* do that?

I had a compiler a while ago (Until SOMEONE around here deleted it.) that would let me decompile .JAR (Java) files and see a lot of cool stuff, like names of function calls and whatnot. I also believe it had a similar function for EXE's but I never tried it.

I don't know what to make of this. It *LOOKS* to be valid but I never heard of a programmer defining variables with Hex. But at the same time, I never programed in "C" before.

What I want to know is, what do the values such as "bp" mean.
EX:

Code: Select all

MOV(bp, sp);
I couldn't find those defined anywhere.



Anyway... about that high resolution patch... :p
"Check out the red marker... it smells like cherries."
User avatar
ckguy
Bipship Engineer
Posts: 1169
Joined: Thu Nov 01, 2007 17:56
Location: Wakefield, RI, US
Contact:

Post by ckguy »

BP and SP are names of cpu registers "base pointer" and "stack pointer" respectively. The stack pointer points to (iirc) the place where the next thing to be pushed onto the stack would go. The base pointer is more general purpose--for example it could point to the zeroth element of some array so you could read/write some element of the array.
User avatar
KeenRush
Android Dummy
Posts: 2560
Joined: Sat Oct 27, 2007 20:57
Location: KEEN1.EXE
Contact:

Post by KeenRush »

That code looks pretty much like it's just translated from assembly to C and some functions are named, I don't think the original one was even slightly looking like that. Yes, it's impossible to get the original one from the assembly instructions, all the function and variable names get lost in compiling. It's going to be extremely difficult to add new features to Keen 1 by using this code.
My newest mod - Commander Keen: Sunset: viewtopic.php?t=8568 | codename H.Y.E.N.A.
User avatar
entropicdecay
Mad Mushroom
Posts: 576
Joined: Thu Nov 01, 2007 15:00
Contact:

Post by entropicdecay »

We still don't know if it compiles, do we? But yeah, it does look difficult to add to.
User avatar
KeenRush
Android Dummy
Posts: 2560
Joined: Sat Oct 27, 2007 20:57
Location: KEEN1.EXE
Contact:

Post by KeenRush »

No, we don't, but I frankly don't even care whether it does or not, this doesn't impress me at all -- I'm 100% sure it's translated by using a program, too, and not made by hand.
My newest mod - Commander Keen: Sunset: viewtopic.php?t=8568 | codename H.Y.E.N.A.
User avatar
ckguy
Bipship Engineer
Posts: 1169
Joined: Thu Nov 01, 2007 17:56
Location: Wakefield, RI, US
Contact:

Post by ckguy »

Lots of the #define's at the top of the file are readable, so this might be of some use.

That Keen1 disassembly that was going on at k:m a while ago (mostly Andy's work, but with some other contributions), how much headway did that make? I don't remember ...
User avatar
Commander Spleen
Lord of the Foobs
Posts: 2384
Joined: Wed Oct 31, 2007 22:54
Location: Border Village
Contact:

Post by Commander Spleen »

There was a Keen 1 disassembly at K:M? I know Andy made some excellent progress with Keen 4, and someone else had completed enough to get a level and a bounder working with the original algorithms. Those were very exciting projects, but obviously too time consuming for those who have the skills.
Gal wrote:What I want to know is how does someone decompile a binary into a language? More specifically... how do *I* do that?
There's more to it than directly converting each instruction into its corresponding keyword in another language. For example, there is a combination of instructions that can be recognised in synergy as a for loop. I don't know much more that I can coherently rant about.
User avatar
entropicdecay
Mad Mushroom
Posts: 576
Joined: Thu Nov 01, 2007 15:00
Contact:

Post by entropicdecay »

Hmm, I didn't know anything about a Keen 1 disassembly over in Keen:Modding. I guess I must've missed it.
User avatar
ckguy
Bipship Engineer
Posts: 1169
Joined: Thu Nov 01, 2007 17:56
Location: Wakefield, RI, US
Contact:

Post by ckguy »

I *might* be confusing this with the Keen 4 disassembly, but I don't think so ...
User avatar
KeenRush
Android Dummy
Posts: 2560
Joined: Sat Oct 27, 2007 20:57
Location: KEEN1.EXE
Contact:

Post by KeenRush »

Yes, Multimania (if I remember correct) has done some Keen 1 disassembling, but I can't remember if it was all Keen 1 already, I think not all, but a large chunk of it anyways. And no, I don't have any link to it...
My newest mod - Commander Keen: Sunset: viewtopic.php?t=8568 | codename H.Y.E.N.A.
User avatar
DaVince
lazy/busy Keener
Posts: 1476
Joined: Thu Nov 01, 2007 15:34
Location: Amsterdam, Netherlands
Contact:

Post by DaVince »

Galaxieretter wrote:What I want to know is how does someone decompile a binary into a language? More specifically... how do *I* do that?
There are programs around. I think one called "hackman" can reverse engineer win32 executables. There must be DOS/Protected mode DOS programs around too if you look around a bit.

By the way, JARs should be easier to reverse engineer... They're only byte compiled .class files packed in a zip file (they renamed the zip extension to jar to fool a bunch of people). Byte compiled files are only half compiled, and you can actually learn to read byte compiled files (though it's not as easy, but then again it's probably easier than assembly) (actually, you could read executable files if you knew what every byte in it means, and thus create tools that reverse engineer it all).

Now that someone has used or created a tool to reverse engineer a horribly coded version of Keen, people can be free to simplify the code and port/create it in Windows, Linux etcetera. This should be a step forward if someone is good enough to be able to interpret and simplify all of this code.
Wow look at me I'm lurking
User avatar
Multimania
Vortininja
Posts: 84
Joined: Sat Nov 10, 2007 8:10
Location: Hiding in a small, cramped corner of the BwB megarocket.
Contact:

Post by Multimania »

Ok. I've taken a brief look at this guy's "decompilation".

At first I was skeptical, noticing some problems in his post on slashdot (the doors are _not_ sprites, they are tiles, but they do spawn these "bodies", something of which I was previously unaware). Looking closely, his code is based on a disassembly and, instead of trying to work out everything each function does in order to recreate the game, he is hooking disbox and replacing bits of the game with direct emulation calls.

From what I have seen, most of his work is accurate, and a bit more complete than my own disassembly, but not very useful to the average patcher and modder. I will, over the (Australian) summer, incorporate all of his code into my disassembly. With this addition bit of work, it may be possible to do a proper, full, compilable decompilation, or, if not, to at least reproduce all of the gameplay code, allowing fangames and remakes to have perfectly accurate physics.

This however is a long term project.

To summarise:
* The code is real, decompiled code, but it will not directly compile.
* The code appears to use some dosbox code to emulate the cpu, allowing the decompiler to not need to analyze all of the code (although a great deal of analysis has been done)
* There are some interesting new breakthroughs that could be found in the code, however the code is not in a state that it could be easily used by modders and patchers (although, again, it could help if you can get around the odd syntax)
* I will work on incorporating these new discoveries into my disassembly, which should be more useful to those who patch.

To summarise this summary:
* Nothing to see here now, but it could spark some interesting breakthoughs in the future.
Post Reply