[v2.11.3] Abiathar: The superior Keen Galaxy editor

[Fleexy]

My goodness, it's been a busy few months. Well, we just celebrated Thanksgiving here in the United States, and what better things are there to be thankful for than Abiathar updates?

UPDATE
v2.2: the audio update

As always, you can get it from the magical auto-updater or in ZIP form.

Let's see what changed!

Well, you might notice immediately, in the download phase, that the Abiathar executable has gotten a lot smaller. Rather than storing the full EGA files in the executable for use as default graphics, it now gets them from GIF tilesheets. With that and the removal of some old resources, the main EXE went from over 6MB to just 3.7MB.

More interestingly, and the big purpose of this update, the Edit menu has gained four new items! First is Audio, which contains sub-items to link your mod's audio resources or dump the default ones for your Galaxy episode. It also lets you Import or Export the resources, smashing them into individual chunks bound together by the *.asnds file. You can export the full ripped chunks (Type-1) or just the sound data, leaving the authoritative metadata in the ASNDS file (Type-0). Either way, the metadata will appear in the ASNDS file for reference or override (replace the "?" at the beginning of the line with an "!" to overwrite the metadata in a Type-1 chunk with the ASNDS metadata when importing).

If you only need to manage the IMF songs, there is a convenient GUI for that! Look no further than Edit | Song Mappings. Once audio resources are linked to the dependency file, you can assign songs to levels - or to phantom level slots, if you need to do some fancy patching. Simply select the level on the left, the song on the right, and click Set Song. If you need more level slots than you have levels, click Set List Length. (You may also have to do this if you set up audio resources separately from the Song Mappings option.) If you want to include more distinct songs than the original game has slots for, no problem! Click Import IMF, choose your file, give the song a name, and assign it to levels like any other. Extra IMF tracks are highlighted by an italicized chunk ID. You can add or remove these extra tracks without worrying about their offsets - Abiathar handles all the patching for you: the level song mappings, the audio files, and the inclusion of extra tracks. Awesome!

Speaking of patching, since Abiathar's Patch Generator will become increasingly important when using its integrated audio management, it has been further empowered. Under Edit | Patches, you can enter your custom patches. Abiathar will add them to the auto-generated patch file so you don't have to manually sync Abiathar-generated stuff with your main file.

Finally, there is an unusual new option that will not be visible to everyone: VeriMaps Signature. VeriMaps is an asymmetric-key crypto-signing system to prove the authenticity of modded level files. If you have a VeriMaps certificate and enable VeriMaps Signature, Abiathar will add a cryptographic signature to the maps file. It looks for the VeriMaps signing certificate (ASIGN file) named VeriMaps.asign in the current directory, or in the location specified in the editor.aconf config option VeriMapsCertPath.

Once a file is signed, its signer's name and special distinction (this is the secret feature I was talking about with the icon artists!) will appear in the status bar when it is opened with Abiathar. For example, opening a file signed by me will produce the text "Signed by Fleexy, creator of Abiathar (VeriMaps confirmed)". If anybody else saves the file and redistributes it, the signature will be destroyed (or will be signed by them). This system can prove who saved a maps file last and help people check that a mirror host did not modify map files from the state intended by the mod author.

You can also check the VeriMaps signature of a GAMEMAPS file with the small command-line utility VeriMaps. Simply drag the maps file onto VeriMaps.exe or launch the utility and type the name of the file and it will check the public key store to verify the signature. If you want to experiment with VeriMaps, you can download the Anonymous signing certificate. I will issue personalized certificates upon request; the file will be sent to you by PM. Keep this file safe - it is your identity in the Abiathar community.

VeriMaps signing works on all level formats except TED5 MapTemp/MapTHead. The signature does not interfere with normal loading by the game.

Small bugs have been fixed and slight speed improvements have been made. Help | About has been added, a short list of acknowledgements. A documentation update is in the works. If there are any questions, refer to this post or talk to me! I'm also soliciting complaints, even minor ones, in the linked K:M thread.

Happy Thanksgiving everyone!

Fleexy

[Keening_Product]

just celebrated Thanksgiving here in the United States

Sorry for going off-topic, but this is more important than anything ever: I THOUGHT YOU LIVED IN THE UK!?!

[Benvolio]

"Blogham Tower" does sound like a neighbourhood you'd find in East London alright.

[MoffD]

just celebrated Thanksgiving here in the United States

Sorry for going off-topic, but this is more important than anything ever: I THOUGHT YOU LIVED IN THE UK!?!

huh, I knew he was in the US but I don't remember where I learned that...

[Roobar]

just celebrated Thanksgiving here in the United States

Sorry for going off-topic, but this is more important than anything ever: I THOUGHT YOU LIVED IN THE UK!?!

huh, I knew he was in the US but I don't remember where I learned that...

You're both wrong. He's from Blogham Tower.

[Fleexy]

Blogham Tower was the spelling of Bloogton Tower that appeared in the Keen 6 betaleak/crack/pirate I used for most of my childhood (and when I registered on this board). Since it was home to several Fleexes and my chosen username was Fleexy, I figured it was appropriate.

And yes, I live in the United States (Midwest).

[Levellass]

That reminds me I have a question about your Keen Dreams maphead fixer, it seems to only work for me if the file is named MAPHEAD.KDR, other files it says are fixed but it doesn't actually alter the file.

Is it even necessary though? Surely Abiathar could detect the not-502 byte maphead and assume brokenness, auto-fixing it without the need of a file to be emitted and whatnot? (If you're really paranoid you can have it ask' Do you want to fix the map header' or somesuch.)

[Fleexy]

Merry Christmas, everyone! There's something under the tree for all of you, and all you have to do to unwrap it is scroll down - it's an Abiathar update!

UPDATE
v2.3: the helpful update

Documentation has been updated as well.

Get it with the auto-updater or download in ZIP form. For those of you running on Linux/Mono, MoffD has figured out how to make the auto-updater work:

Hey everyone!

Just a quick message for Linux users running Abiathar through wine+mono, the updater failed for me and I found a problem some others may experience.

So I got this from running the updater from the cli:

Code: Select all

Error getting response stream (Write: The authentication or decryption has failed.): SendFailure

Which I found to mean that my mono install doesn't have any ssl certs installed. This can be fixed by running:

Code: Select all

sudo mozroots --import --sync

Which will install Mozilla's certificates (Note: mozroots is in the mono-devel packages)

Contextual help has been added! Choose Help | Contextual or just press the question mark key to enable it at any time. The contextual help pane tells you everything your current tool can do at the moment. You might now be able to use the Tile Instance Remapper without constantly poring over the documentation! In my opinion, this change makes Abiathar much easier to use and learn.

I identified a potential problem with the IMF importer in v2.2: if you export all the sounds, modify the IMFs with Music Mappings, and then import all the sounds, the IMF changes will be overwritten unless you also changed the IMF files in the exported sounds. I added an internal "last modified" time for the IMF files so they cannot surprise you by getting overwritten. (To override this behavior, add a "CONFIG FORCEIMF" line to the ASNDS file before the IMFs.)

The Freeform Tile Placer (pencil) tool had a serious problem under a certain configuration state. The issue has been repaired.

The Patches dialog now accepts up to 1MB of text rather than just 32KB. Also, Ctrl+A now works to Select All there.

Some tips have been added to a new "did you know?" pop-up which appears at startup. It can be disabled by unticking its check mark or in the config file.

Several UI glitches involving switching between tools were fixed.

The Tile Instance Remapper has been reworked so as to be a lot more useful and convenient.

That reminds me I have a question about your Keen Dreams maphead fixer, it seems to only work for me if the file is named MAPHEAD.KDR, other files it says are fixed but it doesn't actually alter the file.

Is it even necessary though? Surely Abiathar could detect the not-502 byte maphead and assume brokenness, auto-fixing it without the need of a file to be emitted and whatnot? (If you're really paranoid you can have it ask' Do you want to fix the map header' or somesuch.)

That's been fixed too! The FxTomDMh utility is no longer needed to work with TOM-saved Dreams levels, but I left it in the File Emitter in case somebody wants to keep it around so as to not need to open Abiathar to fix mapheads.

Merry Christmas, everyone! Happy modding!


Fleexy

[Levellass]

Most excellent, I love how this project is moving along.

[troublesomekeen]

Okay. So let's see here. I have an existing project I was working on for Keen Dreams and I want to open it with Abiathar.
First I run the exe.
Click File and New.
The New Project Wizard opens. I can Start from a template or Enter settings manually. There are things in the manual setup I don't understand, like the EGA Layout, so I choose the template option. (This perhaps puts me on the wrong track already.)
Since the game I want to edit is Dreams, the selected Template is Keen Dreams. I suppose I'll leave the Inspect advanced settings box unchecked.
Then I select the Containing Folder. All of the resource files must reside in this folder. So I pick the folder that has my DreamsModProject inside.
Now we're on to the Level Source. I'm not sure whether to choose Load level files from the original game or Start from existing level files. If I can get away with using Abiathar's memory, I'll go with that. So I do.
Now it's on to Level Files. Will Abiathar automatically generate the GameMaps, the MapHead and the MapDict? Or do I have to browse and find these files on my project folder? So I just leave it as is and click next.
Now it's on to Graphics Source. I'm not sure what EGA resources are, and it says the Original game graphics cannot be modified. So that leaves me with Tile sheet bitmaps.

Next it's Graphics Files, Unmasked and Masked. For Unmasked, KDREAMS.EGA is pre-selected. For Masked, EGAHEAD.KDR is pre-selected. So I click next. But Abiathar can't find the EGAHEAD.KDR. And heck if I could. So I click browse and Abiathar demands I take my bitmaps out of my graphics folder (typical TOM set up) and put them into the folder with all the other important resource files (up one level). So now 7TIL0001.BMP is selected for Masked.

On to the Tileinfo Source. I'll want to edit tile properties...with Abiathar? Abiathar can do that? Manage tileinfo as part of the map header looks complex and scary so I'll choose Create or load a separate tileinfo file. Next.

Tileinfo File. Ah. I suppose I choose the Include the original game's tileinfo and leave the KDREAMS.TLI alone. Finished! Congratulations! You have no idea what you're doing! Perhaps you should have finished reading that AbiatharHelp PDF! Oh wait. Error processing dependency file. Save error log? Um. Okay. Sure. I wonder where it saves to. Now Abiathar crashes. Oh dear. This doesn't look good. Close the program.

How about we start all over again? This time we'll try to fix all those dependency things as we go. The GameMaps I have. Yay. What about the MapHead? Uh... I don't see a Maphead. I'll see if I can get away with selecting EGAHEAD.DRE. That oughta hold'em. Wait. MapDict? What is that? There's no way I have a MapDict in my project folder. Looks like this is the end of the road. I'll be dashed if I know where these files come from. I think they're automatically made by modkeen when you import graphics, or you get them from an obscure TOM download. I bet I actually have these troublesome files in a different Dreams project I started years ago. Lemme check my portable harddrive. Yes. Yes, there's EGADICT.KDR. This is with KeenGraph. But the other one was with ModKeen. Let's see if I can copy the EGADICT.KDR over. This is going to end so badly. Probably crash Abiathar again... Oh. Oh! It looks awful! Bleak black darkness everywhere! This is what comes of using KeenGraph things with ModKeen things. At least Abiathar didn't crash.

This has been another chapter from the running diary of TsK.

[Fleexy]

So firstly, this is partially my fault because Abiathar should be crash-proof. (Those error logs that it offered to save usually give great insight into what went wrong in project setup.)

Okay. So let's see here. I have an existing project I was working on for Keen Dreams and I want to open it with Abiathar.
First I run the exe.
Click File and New.
The New Project Wizard opens. I can Start from a template or Enter settings manually. There are things in the manual setup I don't understand, like the EGA Layout, so I choose the template option. (This perhaps puts me on the wrong track already.)

You're OK so far - templates are what should be used unless you're doing something unusual. (The other options are mostly for Lemm.)

Since the game I want to edit is Dreams, the selected Template is Keen Dreams. I suppose I'll leave the Inspect advanced settings box unchecked.

That's fine too - inspecting advanced settings just gives you the same stuff as if you did the manual setup, but filled in with the secret internal sauce that you usually don't have to worry about.

Then I select the Containing Folder. All of the resource files must reside in this folder. So I pick the folder that has my DreamsModProject inside.

You're doing great!

Now we're on to the Level Source. I'm not sure whether to choose Load level files from the original game or Start from existing level files. If I can get away with using Abiathar's memory, I'll go with that. So I do.

Uh oh! Loading level files from the original game means Abiathar will dump out the level files that ship with the game. If this mod was already in progress, you should have opted to start from existing level files (the ones you made with TOM). I believe reading the blob of text near the bottom of the wizard tells you this. (If it doesn't, you're using an old version of Abiathar!)

Now it's on to Level Files. Will Abiathar automatically generate the GameMaps, the MapHead and the MapDict? Or do I have to browse and find these files on my project folder? So I just leave it as is and click next.

Since you chose to start from default level files, the filenames you left there will be used to dump out the default Dreams levels when you save for the first time.

Now it's on to Graphics Source. I'm not sure what EGA resources are, and it says the Original game graphics cannot be modified. So that leaves me with Tile sheet bitmaps.

Oh no, this is where it all started to go bad! You should have chosen EGA resources if you had modified graphics with ModKeen or KeenGraph. EGA resources are the files named KDREAMS.EGA, EGAHEAD.KDR, and EGADICT.KDR. (The ones with "EGA" in the name.)

Next it's Graphics Files, Unmasked and Masked. For Unmasked, KDREAMS.EGA is pre-selected. For Masked, EGAHEAD.KDR is pre-selected.

Oops, that's an Abiathar bug. KDREAMS.EGA is a 100% invalid file to use as the Unmasked bitmap. Since you chose to use tile sheet bitmaps (the things you export with ModKeen or KeenGraph), you need to choose BMP files. (Fun fact: Abiathar can detect whether a bitmap you give it is ModKeen-style or KeenGraph-style.) Those fields should have been blank by default - I'll fix that. This step is almost certainly what caused the load failure.

So I click next. But Abiathar can't find the EGAHEAD.KDR. And heck if I could. So I click browse and Abiathar demands I take my bitmaps out of my graphics folder (typical TOM set up) and put them into the folder with all the other important resource files (up one level). So now 7TIL0001.BMP is selected for Masked.

That's the correct file to use there. Unfortunately, everything is about to go bad because of the Unmasked one.

On to the Tileinfo Source. I'll want to edit tile properties...with Abiathar? Abiathar can do that? Manage tileinfo as part of the map header looks complex and scary so I'll choose Create or load a separate tileinfo file. Next.

Haha, yep, Abiathar has been able to do that since v1.1. Managing tileinfo as a separate resource is what you need to do for Dreams (if you want Abiathar to manage it), so you got that right.

Tileinfo File. Ah. I suppose I choose the Include the original game's tileinfo and leave the KDREAMS.TLI alone.

Uh oh - including the original game's tileinfo, combined with your choice to manage tileinfo as a separate resource (its own file) is going to overwrite whatever is in KDREAMS.TLI with the game's default tileinfo on the first save. If you didn't want Abiathar to manage your tileinfo, you should have chosen "Never use tileinfo" or "Always load the original game tileinfo" on the previous page. There is a helpful text blob there too.

Finished! Congratulations! You have no idea what you're doing! Perhaps you should have finished reading that AbiatharHelp PDF! Oh wait. Error processing dependency file. Save error log? Um. Okay. Sure. I wonder where it saves to. Now Abiathar crashes. Oh dear. This doesn't look good. Close the program.

It seems my directions in the wizard were either not clear enough or not obviously visible. I'll fix that, and the issue with the autofill in the graphics files.

How about we start all over again? This time we'll try to fix all those dependency things as we go.

Persistent! I like it!

The GameMaps I have. Yay. What about the MapHead? Uh... I don't see a Maphead. I'll see if I can get away with selecting EGAHEAD.DRE. That oughta hold'em.

Uh, no, no it won't. Those are completely different types of resources (EGAHEAD.DRE is one of the EGA graphics resources.) What did TOM do with your MAPHEAD? It should be named MAPHEAD.KDR for Dreams.

Wait. MapDict? What is that? There's no way I have a MapDict in my project folder. Looks like this is the end of the road. I'll be dashed if I know where these files come from. I think they're automatically made by modkeen when you import graphics, or you get them from an obscure TOM download.

I don't think ModKeen generated MAPDICT.KDR; TedSetup might. The default is bundled with a certain TOM download, but Abiathar has it inside itself and is able to dump it out if you choose to start from default game levels. How did you manage to save levels with TOM without getting a MAPDICT?

I bet I actually have these troublesome files in a different Dreams project I started years ago. Lemme check my portable harddrive. Yes. Yes, there's EGADICT.KDR. This is with KeenGraph. But the other one was with ModKeen. Let's see if I can copy the EGADICT.KDR over. This is going to end so badly. Probably crash Abiathar again... Oh. Oh! It looks awful! Bleak black darkness everywhere! This is what comes of using KeenGraph things with ModKeen things. At least Abiathar didn't crash.

Putting ModKeen and KeenGraph stuff together usually isn't super bad, but the EGADICT does have to be able to decompress the EGAGRAPH. Did you choose to start from blank level files?

This has been another chapter from the running diary of TsK.

Ouch, man. It seems my in-program documentation is poor. Please try it again after reading my commentary on the problems, and if you still can't get it to work, please send me all the relevant files - I'll check if there's something amiss.

Thanks for trying Abiathar, hopefully we can get this sorted out soon.

Fleexy

[troublesomekeen]

That's superbly helpful. I think we can get through this. It is our destiny. I mean, there are always workarounds, however troublesome they may be. I have sixty bajillion different project folders, some using KeenGraph, others with ModKeen, some only with replaced sprite graphics and no level editing done, others with only level editing in progress. It's really quite a mess. And it doesn't help that I've been out-of-it since Keen Day 2014. Abiathar is super awesome and with this response, everything is much clearer now. I can't complain about the documentation. I was just, like, clicking things. I do have a working Keen Dreams setup (KeenGraph) with Abiathar, as opposed to my ModKeen Dreams setup, which I'd rather not use anyway.

I think the main thing is it's not clear to me what Abiathar will generate and what files it will not.

Oops, that's an Abiathar bug. KDREAMS.EGA is a 100% invalid file to use as the Unmasked bitmap. Since you chose to use tile sheet bitmaps (the things you export with ModKeen or KeenGraph), you need to choose BMP files. (Fun fact: Abiathar can detect whether a bitmap you give it is ModKeen-style or KeenGraph-style.) Those fields should have been blank by default - I'll fix that. This step is almost certainly what caused the load failure.

This what happens when I click Next and Back a few times. Thanks for the goldmine of a response. Thumbs up, thumbs up. Wait. Where be my ThumbsUp-Emotikeen?

[Fleexy]

At the request of Gridlock, I have made several small changes related to music management and patching.

UPDATE
v2.3.1: patching and music adjustments

Get it with the auto-updater!

The Music Mappings dialog now has a button to Rename the song so you don't have to re-import or Replace it to change its name.

There is now a checkbox in the lower-left of the Patches dialog that can be used to disable the overwrite of play.bat when patches are compiled.

Comments have been added to the emitted patch file so you can see what does what.

Gridlock also requested a way to listen to the IMF songs in Abiathar. Sadly, I know nothing at all about audio conversion or playing, but there is a excellent IMF-to-WAV converter. Unfortunately, it is licensed under the GPL, which I do not like at all, and therefore will not be integrated into the main Abiathar program. But! Abiathar has supported extensions/snap-ins/plugins literally forever, so I created an Abiathar extension that lets you preview IMF files in Music Mappings! This way, y'all can have this nice feature and learn how Abiathar extensions work, and I don't have to put Abiathar under the tyrannical GPL.

EDIT: Turns out I didn't apply the GPL correctly, and also I don't have to apply it to ImfPreview because of the way I invoke imf2wav. ImfPreview is not under the GPL, but you can read its code if you like. I will keep this functionality in an extension because I don't want to risk a lawsuit if I'm wrong.

Download ImfPreview for Abiathar v2.3.1 (source included)

Hopefully this will continue to work in future versions of Abiathar, but I did a lot of reflection/code-injection stuff that may or may not keep working if Abiathar's internals change significantly. To use, simply extract the ZIP and copy ImfPreview.aex to the directory containing Abiathar. Run Abiathar as normal; you will see an Extensions menu added if it was loaded successfully. Right-click songs in the right-hand list of Music Mappings to hear them.

Happy modding!

Fleexy

[Fleexy]

Hey TK, this one's for you!

UPDATE
v2.3.2: the confusion-removing update

Get it with the auto-updater!

The New Project Wizard now has an option to just prompt for the four simple resources - GameMaps, MapHead, background tileset, foreground tileset - for those who are used to TOM and don't need all the superpowerful features of the normal NPW. It will attempt to auto-detect all the other resources after you select the GameMaps, but doesn't usually find bitmaps named in the KeenGraph style. (Both bitmap formats work, of course.) If you later decide you need something like tileinfo, you can use Edit | Project Settings to access the big NPW. Audio and Infoplane Override can be configured as well, after the fact.

I realized that putting all your bitmaps in the game directory could get messy and inconvenient, so I allowed all resources to be loaded from a subdirectory (no matter how deep, as long as the ADEPS file is in a directory that is an ancestor of it). So, "BMP\4TIL0000.BMP" and "SFX\TEMP\AUDIO.CK6" are acceptable paths. (Things like "..\MAPHEAD.CK4" or "\Users\Fleex\Documents\KDREAMS.EGA" are not.) The browse-file buttons all determine the relative path of your selection; you are also free to type a relative path directly into a textbox.

Hopefully this update makes things more convenient. Happy modding!

Fleexy

[MoffD]

Hmm... "Error loading extensions!" when I put the imf preview extension in the same directory. Does Abiathar have an error log I can check, or possibly a verbose/debugging mode?

This is after the update to Abiathar though...