AUR Support for Arch users

All discussion about the Commander Genius project (a Keen interpreter + more).
Post Reply
gerstrong
Vorticon Elite
Posts: 1143
Joined: Wed Dec 31, 2008 14:44
Location: Frankfurt - Germany
Contact:

AUR Support for Arch users

Post by gerstrong » Sat Dec 16, 2017 13:47

Hello,
finally I got CG working with AUR:

https://aur.archlinux.org/packages/comm ... enius-git/

For those who are experienced with Linux and like the Arch based distros, this is for you. Just open AUR for the packages and let it build.

Have fun!
Having fun developing stuff...

User avatar
Nisaba
Vorticon Elite
Posts: 822
Joined: Fri Jan 01, 2016 23:34
Location: The Outpost
Contact:

Re: AUR Support for Arch users

Post by Nisaba » Sat Dec 16, 2017 14:11

great! as soon as I checked things out, I'll let you know.
"We have come to conquer you all and bring you lots of candy!" | about me | my current project: play Keen mods on a GameBoy |

User avatar
Malvineous
Shikadi Webmaster
Posts: 362
Joined: Wed Oct 31, 2007 21:48
Location: Brisbane, Australia
Contact:

Re: AUR Support for Arch users

Post by Malvineous » Sun Dec 17, 2017 1:47

Thanks for this, I was able to install it, although it did a full git clone rather than a shallow one (--depth 1) so it took a long time to download thanks to my slow Internet connection.

I noticed that the compile paths (pkg and src) seem to be hard-coded into the binary which is not so useful after the install:

Code: Select all

n: Looking at: /usr//commandergenius/gameCatalogue.xml
n: Looking at: Commander-Genius/gameCatalogue.xml
n: Looking at: pkg/gameCatalogue.xml
n: Looking at: src/gameCatalogue.xml
But it doesn't cause any problems. I fed it my Keen directory and it recognised many of the versions, but a few are unknown so I will list them here just in case you're interested in adding support. Probably the only real issues are that it looks like Keen Dreams v1.13 is detected as v1.0, the real v1.0 shows up as an unknown version, and the latest version 1.93rev1 is unrecognised. The Keen 4 demo also isn't recognised (perhaps because it uses KEEN.EXE instead of KEEN4E.EXE).

Code: Select all

n: EXE processed with size of 100484 and crc of 586BE7FA
n: Detected game Name: games/1-131/keen1.exe Version: v1.31

n: EXE processed with size of 118672 and crc of BAFF21E8
n: Detected game Name: games/2-131/keen2.exe Version: v1.31

n: EXE processed with size of 127616 and crc of F5D8BD71
n: Detected game Name: games/3-131/keen3.exe Version: v1.31

n: EXE processed with size of 263488 and crc of DD8DB331
n: Detected game Name: games/4-14/keen4e.exe Version: v1.40

n: Search: games/4-demo

n: EXE processed with size of 266096 and crc of 4BEAD24D
n: Detected game Name: games/5-14/keen5e.exe Version: v1.40

n: EXE processed with size of 238368 and crc of 861A34E9
n: Detected game Name: games/6-promo/keen6.exe (Unknown Version)

n: EXE processed with size of 256656 and crc of 61AAA16F
n: Detected game Name: games/6c-10/keen6.exe (Unknown Version)

n: EXE processed with size of 266032 and crc of 4A35EE01
n: Detected game Name: games/6e-10/keen6.exe (Unknown Version)

n: EXE processed with size of 236112 and crc of 3205AC28
n: Detected game Name: games/6e-10dmo/k6demo.exe Version: v1.0

n: EXE processed with size of 238368 and crc of 861A34E9
n: Detected game Name: games/6e-10pmo/keen6.exe (Unknown Version)

n: EXE processed with size of 77694 and crc of B9FC3780
n: Detected game Name: games/d-10/kdreams.exe (Unknown Version)

n: EXE processed with size of 213424 and crc of 19B65939
n: Detected game Name: games/d-101-sw/kdreams.exe (Unknown Version)

n: EXE processed with size of 213536 and crc of 8981185D
n: Detected game Name: games/d-113-sw/kdreams.exe Version: v1.0

n: EXE processed with size of 212272 and crc of B717D006
n: Detected game Name: games/d-192r0/kdreams.exe (Unknown Version)

n: EXE processed with size of 213200 and crc of 1D639DA9
n: Detected game Name: games/d-193r1/kdreams.exe (Unknown Version)

n: EXE processed with size of 79360 and crc of C6E3059D
n: Detected game Name: games/fixed-beta/keen1.exe (Unknown Version)

n: EXE processed with size of 263489 and crc of E76BF416
n: Detected game Name: games/mods/birdman/keen4e.exe (Unknown Version)
I haven't had a chance to play CG for a very long time so this is pretty much the first time I have run it since CKP! I find it a bit difficult to control as the physics are so different to the original games though - is the idea to still have a clone of the originals or is it just to get it close enough? Quickly playing Keen 4 though CG I can see things like the gravity is higher and movement speeds are different, you have to be exactly in the right spot when you press the up arrow to enter doors (the original games let you be one tile to the left or right and Keen slides across), the Bounder doesn't follow the player, etc. But I'm not sure how accurate the goal is to get to the original games. I also sat there for ages wondering why it was taking so long to load the levels before I realised you have to press enter to make the 'loading' screen go away!

Having said that it's a fantastic achievement and it works really well, so very impressive.

Also I'm curious - the main page still seems to be on SourceForge but the source is on GitHub - are they both being used or is one preferred to the other?

gerstrong
Vorticon Elite
Posts: 1143
Joined: Wed Dec 31, 2008 14:44
Location: Frankfurt - Germany
Contact:

Re: AUR Support for Arch users

Post by gerstrong » Sun Dec 17, 2017 11:37

Thanks for your comments Malvineous, as usual always helpful nice hints you give me. Let's break down your text.
Malvineous wrote:
Sun Dec 17, 2017 1:47
Thanks for this, I was able to install it, although it did a full git clone rather than a shallow one (--depth 1) so it took a long time to download thanks to my slow Internet connection.
This a problem I have to tackle with Git. We have been moving too much around over the years. There is a way to hoover a bit the repository. Not sure yet how. If everything fails we create open a new one.
Malvineous wrote:
Sun Dec 17, 2017 1:47

I noticed that the compile paths (pkg and src) seem to be hard-coded into the binary which is not so useful after the install:

Code: Select all

n: Looking at: /usr//commandergenius/gameCatalogue.xml
n: Looking at: Commander-Genius/gameCatalogue.xml
n: Looking at: pkg/gameCatalogue.xml
n: Looking at: src/gameCatalogue.xml
But it doesn't cause any problems. I fed it my Keen directory and it recognised many of the versions, but a few are unknown so I will list them here just in case you're interested in adding support. Probably the only real issues are that it looks like Keen Dreams v1.13 is detected as v1.0, the real v1.0 shows up as an unknown version, and the latest version 1.93rev1 is unrecognised. The Keen 4 demo also isn't recognised (perhaps because it uses KEEN.EXE instead of KEEN4E.EXE).
I don't get this really. CG provides a search path system you can configure yourself. Some standard paths are hardcoded in case the user decides not to configure anything. But you can add others search paths as well. The hardcoded paths depend on the operating system to use. The path "/usr/" is typical for Linux systems, but that in only one of the paths the system looks into.
Malvineous wrote:
Sun Dec 17, 2017 1:47

Code: Select all

n: EXE processed with size of 100484 and crc of 586BE7FA
n: Detected game Name: games/1-131/keen1.exe Version: v1.31

n: EXE processed with size of 118672 and crc of BAFF21E8
n: Detected game Name: games/2-131/keen2.exe Version: v1.31

n: EXE processed with size of 127616 and crc of F5D8BD71
n: Detected game Name: games/3-131/keen3.exe Version: v1.31

n: EXE processed with size of 263488 and crc of DD8DB331
n: Detected game Name: games/4-14/keen4e.exe Version: v1.40

n: Search: games/4-demo

n: EXE processed with size of 266096 and crc of 4BEAD24D
n: Detected game Name: games/5-14/keen5e.exe Version: v1.40

n: EXE processed with size of 238368 and crc of 861A34E9
n: Detected game Name: games/6-promo/keen6.exe (Unknown Version)

n: EXE processed with size of 256656 and crc of 61AAA16F
n: Detected game Name: games/6c-10/keen6.exe (Unknown Version)

n: EXE processed with size of 266032 and crc of 4A35EE01
n: Detected game Name: games/6e-10/keen6.exe (Unknown Version)

n: EXE processed with size of 236112 and crc of 3205AC28
n: Detected game Name: games/6e-10dmo/k6demo.exe Version: v1.0

n: EXE processed with size of 238368 and crc of 861A34E9
n: Detected game Name: games/6e-10pmo/keen6.exe (Unknown Version)

n: EXE processed with size of 77694 and crc of B9FC3780
n: Detected game Name: games/d-10/kdreams.exe (Unknown Version)

n: EXE processed with size of 213424 and crc of 19B65939
n: Detected game Name: games/d-101-sw/kdreams.exe (Unknown Version)

n: EXE processed with size of 213536 and crc of 8981185D
n: Detected game Name: games/d-113-sw/kdreams.exe Version: v1.0

n: EXE processed with size of 212272 and crc of B717D006
n: Detected game Name: games/d-192r0/kdreams.exe (Unknown Version)

n: EXE processed with size of 213200 and crc of 1D639DA9
n: Detected game Name: games/d-193r1/kdreams.exe (Unknown Version)

n: EXE processed with size of 79360 and crc of C6E3059D
n: Detected game Name: games/fixed-beta/keen1.exe (Unknown Version)

n: EXE processed with size of 263489 and crc of E76BF416
n: Detected game Name: games/mods/birdman/keen4e.exe (Unknown Version)
There is a CRC checker for EXE in order to get sure, that the games are supported. I have never tested Keen 4 Demo so it is not supported. I consider that one very rare. The Keen 6 Demo was also not supported, but the last two month I got some help and now it works! At least one way to play some Keen 6 levels at no cost.

Malvineous wrote:
Sun Dec 17, 2017 1:47
I haven't had a chance to play CG for a very long time so this is pretty much the first time I have run it since CKP! I find it a bit difficult to control as the physics are so different to the original games though - is the idea to still have a clone of the originals or is it just to get it close enough? Quickly playing Keen 4 though CG I can see things like the gravity is higher and movement speeds are different, you have to be exactly in the right spot when you press the up arrow to enter doors (the original games let you be one tile to the left or right and Keen slides across), the Bounder doesn't follow the player, etc. But I'm not sure how accurate the goal is to get to the original games. I also sat there for ages wondering why it was taking so long to load the levels before I realised you have to press enter to make the 'loading' screen go away!
Wow, that has been some time. Thanks for taking a second look. The controls are hard, because we have different timers running, which are not the original ones. I want to replicate the feeling. I think in Keen 1-3 it is better. So I might try this with some dosbox comparison when I get more time. I want to take what is good from the original games. I want to leave out, what is bad, or make it optional. And I want to improve physics and feeling a bit more. That's the goal.

I have reported the Doors thing in Github. That will be changed.
Malvineous wrote:
Sun Dec 17, 2017 1:47
Having said that it's a fantastic achievement and it works really well, so very impressive.
Thank you very much. :-)
Malvineous wrote:
Sun Dec 17, 2017 1:47
Also I'm curious - the main page still seems to be on SourceForge but the source is on GitHub - are they both being used or is one preferred to the other?
Sourceforge is website- and fileserver in my book. Github stores the source code. SF doesn't have a good GUI for Project management in my opinion, that is why Github is being used. It was an idea of Albert and I like it. Before, when SVN was used, everything was hosted on SF.
Having fun developing stuff...

User avatar
Nisaba
Vorticon Elite
Posts: 822
Joined: Fri Jan 01, 2016 23:34
Location: The Outpost
Contact:

Re: AUR Support for Arch users

Post by Nisaba » Mon Dec 18, 2017 1:02

so I temporary temporarily killed my ArchLinux machine.
need.to.fix.things.first.
plus I guess I cannot share much more detailed info than Malv already provided.
"We have come to conquer you all and bring you lots of candy!" | about me | my current project: play Keen mods on a GameBoy |

User avatar
Malvineous
Shikadi Webmaster
Posts: 362
Joined: Wed Oct 31, 2007 21:48
Location: Brisbane, Australia
Contact:

Re: AUR Support for Arch users

Post by Malvineous » Mon Dec 18, 2017 12:19

gerstrong wrote:
Sun Dec 17, 2017 11:37
This a problem I have to tackle with Git. We have been moving too much around over the years. There is a way to hoover a bit the repository. Not sure yet how. If everything fails we create open a new one.
I think it is good to preserve the history, and I did some investigation and it looks like it's a limitation with Arch's makepkg more than anything else. Seems the devs always clone the entire history so they can jump back to old commits to check for bugs, so they declare that everyone else must want to do the same also. Not much you can do when the devs are so against an idea it seems.
gerstrong wrote:
Sun Dec 17, 2017 11:37
I don't get this really. CG provides a search path system you can configure yourself. Some standard paths are hardcoded in case the user decides not to configure anything. But you can add others search paths as well. The hardcoded paths depend on the operating system to use. The path "/usr/" is typical for Linux systems, but that in only one of the paths the system looks into.
It's no big deal, I just thought it odd that it looked for a folder called "pkg" and another called "src" in the current directory.
gerstrong wrote:
Sun Dec 17, 2017 11:37
Wow, that has been some time. Thanks for taking a second look. The controls are hard, because we have different timers running, which are not the original ones. I want to replicate the feeling. I think in Keen 1-3 it is better. So I might try this with some dosbox comparison when I get more time. I want to take what is good from the original games. I want to leave out, what is bad, or make it optional. And I want to improve physics and feeling a bit more. That's the goal.
It was too long! :) I was thinking about this, and I wonder whether it would be useful to implement demo playback in CG? If you could record a demo in the original game and play it back in CG, then the physics would have to be spot on for the demo to work, otherwise the player would likely die or get stuck very quickly. I made use of this when I ported Xargon as it was a very good way to check that everything was working at the proper speeds. Just ask KeenRush to record the demo and you'll only be able to play it back in CG successfully if your physics match the originals 100% :lol
gerstrong wrote:
Sun Dec 17, 2017 11:37
Sourceforge is website- and fileserver in my book. Github stores the source code. SF doesn't have a good GUI for Project management in my opinion, that is why Github is being used. It was an idea of Albert and I like it. Before, when SVN was used, everything was hosted on SF.
Ah I see. You know you can use GitHub Pages to serve a web site too? You can keep the web site code in your git repo, so it gets version control too. GitHub even lets you host the site from your own domain, if you ever felt like putting something up at commandergenius.org and not having to pay for separate web hosting. But that explains why, thanks for that!

User avatar
K1n9_Duk3
Vorticon Elite
Posts: 474
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

Re: AUR Support for Arch users

Post by K1n9_Duk3 » Mon Dec 18, 2017 22:15

Malvineous wrote:
Mon Dec 18, 2017 12:19
If you could record a demo in the original game and play it back in CG, then the physics would have to be spot on for the demo to work, otherwise the player would likely die or get stuck very quickly. I made use of this when I ported Xargon as it was a very good way to check that everything was working at the proper speeds
I did the same thing while working on ReDuke and other projects, and I still think this is the best way to check for errors in the new implementation.

But for the Keen games, there's a much easier solution at hand. If I'm not mistaken, Chocolate Keen and Omnispeak are 100% faithful recreations of Keens 1-6, and the original source code for Keen Dreams is also available. NY00123 mentioned that the demo playback was used in the recreations to check if they are accurate.

If you want 100% authentic gameplay in Commander Genius, then you could just check out the source code of these projects and merge the game logic into Commander Genius. There's really no need to re-invent the wheel here. This is actually one of the reasons why I stopped working on ReDuke. I realized that at some point I won't be able to get even closer to a 100% accurate reimplementation unless I disassemble the original executables and recreate the original code based on that disassembly.

You're lucky to be in a position where you don't need to deal with the disassembly yourself. Somebodey else already did all the hard work.

On the other hand, I can understand if you don't want to trash most of the code you've written for CG in the last nine years. You would probably have to re-implement many of the features that are already in CG.
Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net

gerstrong
Vorticon Elite
Posts: 1143
Joined: Wed Dec 31, 2008 14:44
Location: Frankfurt - Germany
Contact:

Re: AUR Support for Arch users

Post by gerstrong » Wed Dec 27, 2017 9:19

@K1n9_Duk3: As usual thanks for your hints I really appreciate the time you take for judging the project. Getting to what you wrote here:
K1n9_Duk3 wrote:
Mon Dec 18, 2017 22:15
Malvineous wrote:
Mon Dec 18, 2017 12:19
If you could record a demo in the original game and play it back in CG, then the physics would have to be spot on for the demo to work, otherwise the player would likely die or get stuck very quickly. I made use of this when I ported Xargon as it was a very good way to check that everything was working at the proper speeds
I did the same thing while working on ReDuke and other projects, and I still think this is the best way to check for errors in the new implementation.
This is a good idea, but CG is not able to read demos. I wanted to create an own file format for that, but for such idea it might be better to read those demos. The data format of the demos is known if I am not mistaken.
K1n9_Duk3 wrote:
Mon Dec 18, 2017 22:15

But for the Keen games, there's a much easier solution at hand. If I'm not mistaken, Chocolate Keen and Omnispeak are 100% faithful recreations of Keens 1-6, and the original source code for Keen Dreams is also available. NY00123 mentioned that the demo playback was used in the recreations to check if they are accurate.
I have taken from the Chocolate Keen and Omnispeak codes with some modifications here and there. That code does not work as is, because it is meant to played at lower frame rates, still a lot is based on them. I have 70 ponder per second in the Galaxy games, the original uses 14. It is a good factor, thanks to NY00123 (I think he had the idea) but still you have to adapt things here and there. The Omnispeak code taken was from an older version. It really pushed CG to get Keen 5 playable. I might take a second look here and there. Also special thanks to Pickle for the file format of Keen 4.
K1n9_Duk3 wrote:
Mon Dec 18, 2017 22:15

If you want 100% authentic gameplay in Commander Genius, then you could just check out the source code of these projects and merge the game logic into Commander Genius. There's really no need to re-invent the wheel here. This is actually one of the reasons why I stopped working on ReDuke. I realized that at some point I won't be able to get even closer to a 100% accurate reimplementation unless I disassemble the original executables and recreate the original code based on that disassembly.
That is exactly the thing it worries. Many fans of CG do not care that much as you do, Billy Blaze is there you can play the games, everything fine. I still want to get a bit closer, but smooth animations, some better logic here and there is what I would prefer. I can improve the logic, but I do not want to go down into disassembly. And I don't think it is required. Every update has better physics. Every given hint is taken seriously and implemented with the best of my skills into CG.
K1n9_Duk3 wrote:
Mon Dec 18, 2017 22:15

You're lucky to be in a position where you don't need to deal with the disassembly yourself. Somebodey else already did all the hard work.
That is the true. My only problem is that it got hard to find more time getting the stuff into CG. Years ago there was much more help than now, especially in the hacking department. Many people outside do not see the requirement to get the physics better, it is more about more games, multiplayer, etc. I think regarding game logic we still can improve a lot. So this Forum and the Modding one really have the experts who care. But I am not such an expert for making the game as keenish as many would prefer from such perspective. There are other priorities I have to consider right now.

I also think we should move forward. The patches are nice for changing the game and so on, but I have created a python interface which I think is better for mods. I also have the vision to get rid of any EXE and Assets file dependencies, so we can finally create our own Keen game, maybe with another name on it, so no more licensing issues, maybe something like Freedoom but for Keen. So far at least, there is the mod of Nisaba called "Buried in Oblivion". Also Keen 9 uses some python code to get it running. I have been thinking about interpreting patch code segments, but I always get the sensation it limits us too much. Python is readable to say the least.

I also find a bit sad that mods like "Marooned on Mars" Galaxy Edition and Keen 9 require Keen 5 meaning you can only play it if you bought the game. That is the reason why I do not provide them for download. Yes I know there are other ways to solve this problem and provide those downloads, but it does not change the fact that you need to buy the game. Wouldn't it be cooler not to require the EXE files, have some readable (Python) code for your game logic and your own datasets? I have seen so many mods up to the points, so much creativity, but the EXE limitation and Dosbox I think hurts us a lot. Just my opinion. I respect the nostalgia you feel playing it in Dosbox and I hope to get CG to the point you would feel the same one day...
K1n9_Duk3 wrote:
Mon Dec 18, 2017 22:15
On the other hand, I can understand if you don't want to trash most of the code you've written for CG in the last nine years. You would probably have to re-implement many of the features that are already in CG.
Yes it was a lot of work. Nonetheless I see that a lot of stuff could have been better and still can be, but I also had to learn making the engine better. The latest commits have been bug fixes and cleanups so the Release 2.0 would be much better in many regards. After that release we might take a second look and maybe someone wants to give a helping hand making physics better. I really would like to see it, because with CG we can make mods even more popular getting them playable with more native code and less limitations.

... or maybe we should let Billy Blaze go ...
Having fun developing stuff...

User avatar
K1n9_Duk3
Vorticon Elite
Posts: 474
Joined: Mon Aug 25, 2008 9:30
Location: Germany
Contact:

Re: AUR Support for Arch users

Post by K1n9_Duk3 » Wed Dec 27, 2017 22:13

I'm pretty sure that this community will always notice all the small differences between CG gameplay and original gameplay. And that is to be expected, after decades of playing Keen 1-6 (not to mention all the mods). This might be part of the reason why people are still patching the original executables instead of creating mods with Commander Genius.

Another reason might be conveniance. Most of the active modders have already released at least one mod for the original games. And for those who haven't, there are tutorials on how to set things up (on the KeenWiki, for example). I'm actually not sure if creating custom levels for CG would actually be faster or easier then creating custom levels for the original games.

And actual modding... yeah, I can totally see the appeal of using Python (or other scripting languages) instead of having to write machine code by hand (or letting levellass write machine code by hand) to change the behavior of the in-game objects. The problem is: I'm not much of a modder or level designer. And I guess most modders are outsourcing the majority of the programming/patching work (to levellass) anyway, so there is little motivation to actually learn Python and do the programming themselves.

I guess you'll need a really good mod/fangame that demonstrates all the superior abilities of CG, if you want to motivate people to design mods specifically for CG. Something similar to Keen 7, which (I think) was THE project that made the Keen modding community what it is today. But this also means that you'll either have to do it yourself or at least convince a talented modder to do the design while you take care of the programming/scripting. Otherwise it's not gonna happen.

I know how you feel, believe me. I've been working on KEENGINE for almost 10 years now. It was used in a few short fangames, but those weren't big hits. And all of them were either created by Roobar (a.k.a. wiivn) or myself.
Hail to the K1n9, baby!
http://k1n9duk3.shikadi.net

User avatar
MoffD
Vorticon Elite
Posts: 1169
Joined: Thu Jul 05, 2012 17:30
Location: /dev/null
Contact:

Re: AUR Support for Arch users

Post by MoffD » Mon Jan 08, 2018 16:41

I missed this somehow, thanks gerstrong!
mortimermcmirestinks wrote: Now I wish MoffD wasn't allergic to me.
Levellass wrote:You're an evil man.
Image

Post Reply