Oh, cool! Thanks for the notes; I'll integrate them in with my docs soonish; I've been meaning to implement some things for the next version of the editor but haven't actually taken the time to work on it for awhile now (damn Minecraft, damn Fathamurk...) Anyway, I appreciate it; especially the work you've put into deciphering some of those unknown values. It's been awhile since I've looked at those.

Thanks again!

(Oh, and at this point I'd be remiss if I didn't thank SpottedShroom for his additional scripting notes as well, sent to me seemingly ages ago. I'll get those integrated soon too, I swear. :)

I just tried actually installing and running the character editor and noticed that... well... you can't actually use it. At least, not for what I intended, which was to give myself a bunch of skill books and see what changed in the save file when I read one.

Far as I can tell, you can't just select an item type to plonk into a slot: you have to manually edit every field. Bleegh. But I might be missing something in the UI.

In the exe file, beginning around 13EF9C, there's a "master_item_sheet.csv" table of all the default items in the game.

Might be worth having your code parse that and use it to give users a picker where they can grab any item prefab they want.

Since the exe may change with updates, I'd not rely on the address, but rather just scan forward for the table header, which is the string: "ITEM CATEGORY","DESCRIPTION","LBS","SKILL REQUIRED","RARE","ICON","$$","COMB","DAM","AR","ATTR","AT m","SKILL","SK m","HP m","MANA m","2H m","DMG m","AR m","RES m","Effect","Script (called when USED)".

But ideally, there will be a better way to extract the embedded resources in the exe. Right after that table is the string "master_item_sheet.csv" (filename is in unicode, so search for i\x00n\x00c\x00... etc) - which I assume is the name of the original embedded file that contained that file.

If you can extract.... you can maybe embed, too. And that will give your editor control over conversation trees, and book contents, and so much more. Translations. Grammar/spelling fixes. New NPCs. You name it.

Immediately after that in the file is a bunch of monster stats in CSV, with the filename "incbin/entities.csv" (handy for making a monster picker in the map editor! Or if you can edit it, making new monsters!), then a bunch of texts, with the filename "incbin/books.txt" - very handy if the user is picking a book, maybe? And if you can edit it, for making books!

After that is a bunch of conversation tree files, each called something like "incbin/87.tre". Decoding the conversations could be fun, as they contain scripts too. But making it so that new .tre files can be created, that's the ultimate.

Hey, Googling "incbin blitzbasic" tells me this is a standard thing with BB. So, maybe we can extract, modify, and reinsert. However, can't find anything on their forums on extracting, other than a link to a program that extracts standard format files that are embedded in other files, but isn't any use to us, since it doesn't have plugins for any of the filetypes we're interested in.

I guess, just need to do it the hard way then. Need to OK it with BG of course. But they've been good about the editor so far.

These are the ones that seem to be in the exe:
incbin/51.tre
incbin/52.tre
incbin/53.tre
incbin/54.tre
incbin/55.tre
incbin/56.tre
incbin/57.tre
incbin/58.tre
incbin/59.tre
incbin/61.tre
incbin/62.tre
incbin/63.tre
incbin/64.tre
incbin/65.tre
incbin/66.tre
incbin/67.tre
incbin/68.tre
incbin/69.tre
incbin/70.tre
incbin/71.tre
incbin/72.tre
incbin/73.tre
incbin/74.tre
incbin/75.tre
incbin/76.tre
incbin/77.tre
incbin/79.tre
incbin/80.tre
incbin/81.tre
incbin/82.tre
incbin/83.tre
incbin/85.tre
incbin/86.tre
incbin/87.tre
incbin/88.tre
incbin/89.tre
incbin/90.tre
incbin/91.tre
incbin/92.tre
incbin/93.tre
incbin/94.tre
incbin/95.tre
incbin/96.tre
incbin/books.txt
incbin/celtic.ttf
incbin/entities.csv
incbin/f1.ttf
incbin/kelt.ttf
incbin/master_item_sheet.csv
incbin/narrative.txt

Filename seems to come AFTER the body of the file, and is in unicode.

The incbin data seems to be after all other executable content in the exe, so editing these files MAY be as simple as figuring out where the length byte is before each file, and changing that.

Some random ponderings:
Is the character "$" translated by message(str) to the player's name? Or is that just conv trees? Does prefixing a word with * ^ & or $ make it do anything special, or is that just in skill descs?

What are the numbers from the commands "narrative N" and "notebox N" from?

Also, do we know what affects the verb? That is, whether you EAT, DRINK, READ, or USE an item that has a script? Is that governed by the item type?

I'm only writing all this because I'm slightly delusional from a cold, waiting for my fever to break so I can sleep. I'm too out-of-it to do any real work, I can't concentrate at all, but flow-of-consciousness poking at a game seems quite possible and is better than lying awake looking at the ceiling. So, once I'm better, odds are I won't post again.

Kill_quest displays the message "A quest in your Quest Journal has been removed because it can no longer be finished. You may still be able to complete the game." - so if you want to silently disable a quest, better to set it to stage -1?

Ooh, turns out you can have custom ports. Didn't see that mentioned in the manual, but yeah - stick "mypic.png" (60x60) in the main app folder, and it'll pick it up next time you create a character. And, hey, the character editor already knows about those and respects them! Nice. Way ahead of me as usual.

5:30am. Off to bed.

That's largely right - if you expand the thingy next to the items in the main list, you do have the ability to at least copy/paste items, so you can clone items pretty easily and then just modify the single fields that you're interested in. I would be the first to agree that the UI is hardly ideal; I'm a commandline guy at heart, and I would never list UI design amongst my strengths. :)

Ah, nifty. I've specifically avoided trying to read anything out of the actual game executable, though, and I'm pretty unlikely to do so in the future. I'd have to maintain an understanding of the executable on three different platforms (linux/mac/win), and then Windows of course is bundled by different content distribution networks as well, which could change things around. In the end I'm just not interested in maintaining that kind of thing (and I wouldn't want to distribute an extracted CSV with the util; I'd like to keep the editor as "clean" as possible).

As you mention, I could certainly scan for a header, but I'm afraid I just don't have much interest in doing so. :)

I have occasionally considered just making up my own item prototypes that you could select from a dropdown or something, and that may end up happening one of these days. Perhaps I'll code up a simple class that could be used to store premade objects and call out for submissions, rather than doing all the drudgework myself.


Yeah, though once again, I don't have any interest in my util parsing content in the executable itself, and I don't want to redistribute content embedded in there, either. (Not to mention that actually editing that information would be problematic at best; I'd only really be able to do a readonly view of those things, which doesn't strike me as worth the effort.) For what it's worth, BW has mentioned a few times that he'll probably release his own game editing tools when Book 3 comes out (or at least at some point after Book 3 is out), at which point we'd have a more comprehensive set of tools available to work on things like conversations, etc).

Get better! And thanks for the tips re: scripting.

I'm pretty sure that for blitz-basic, platform-dependent differences in the exe will be at worst a minor issue. I've confirmed the same basic exe layout for E1 in Linux, Mac and Windows. The script below works to extract the files and strings for any of them.

Differences between E1 and E2 are more likely to be an issue than differences between OSs. *checks* Yeah - E2 has nothing useful included in the exe - odds are the good stuff's in "datapak", which looks encrypted apart from the filelist at the end. Well, poo, that ruins that plan then.

Still, for anyone who cares, or is interested in learning from the scripts in the conversation trees, you can use this script to extract the embedded files and unicode strings in both E1 and E2 under any OS, though exe filenames may need changing. Script is php as my python-fu is weak. Run it in the same folder as your executable.

Legal

The E1 software license says "* Limitations on Reverse Engineering, Decompilation and Disassembly. The Software in both object code and source code form includes valuable trade secret information of Basilisk Games. You may not make any copies of the Software beyond the number necessary to exercise your license rights in the Software, and shall not provide copies of the Software to any third party. You may not reverse engineer, decompile, or disassemble the Software or otherwise attempt to gain access to the source code for the Software. "

My reading of this is that program code is "source code for the software", and must not be touched, but embedded data is just data, fine to extract for personal use and modding, but obviously still subject to copyright. However, IANAL: consult one if concerned, or ask BG. In particular, scripts seem like a greyish area that might invalidate your license.

Etc

The above code assumes that the beginning of the file won't look like Unicode. That's true in the case of all the embedded files, but a more rigorous script would parse the "string length" int, and not grab anything more into the strings than it needed to. Because I was using regexes, I didn't bother, so a very few of the strings in the extracted UnicodeStrings.txt table have one or two garbage characters on the end, like: "Weapon, Thrown$", "You are cursed!!i", "response_marker.png$", etc.

Looks like I was WRONG about the embedded files being the last stuff in the exe, plus I can't find any allocation table for resources anyway, both of which mean that changing the length of any of this stuff will unfortunately be an unholy nightmare. Still, data within the files should be able to be changed freely, so long as the length remains. So conversation options COULD be added, and typos fixed, at the cost of editing some other conversation items to be shorter.

[Edit: Another assumption I made in the above scripts is that files are separated by unicode strings. This is only mostly true. They are terminated by unicode strings, and they come one after the other, but there's no leading marker. So, the very first file will have a few bytes of leading junk. Except, we know that the first file, in both cases, is a font file, and we know what sequence font files begin with, so we can strip the leading bytes with a reasonable degree of confidence. I just updated the code to do that properly on all OS's: Linux and Mac had more leading junk than windows, but this fix should resolve that.]

[Edit2: I'd been assuming that strings were 255 characters or less, which is false. Changed the assumption to 64k characters or less, which is true.]

Decoding the .tre files
Here's what I've got, which isn't much. I write ints as [NN] for brevity, eg "0x0A 0x00 0x00 0x00" = "[0A]". Anywhere you see [nn], I mean a 4-byte int.

Each NPC has a .tre file named with the NPC's entity number. So Gunther (entity 66) has his conversation tree stored in the file "66.tre".

Conversation strings are stored as a series of string pairs, which I'll call "exchanges", where the string the player can say is immediately followed by the NPC's response.

The exchange is preceded by two ints.

The first, int1, is the exchange id, which can be anything so long as it's unique within the file. Typically, it starts at 0 and increments for each exchange, but it doesn't need to: they are sometimes randomly ordered in the file, with gaps. This is typically [00] for the initial greeting, which is played only once, when you first meet the character, and [01] for the subsequent greeting that you see any other time, though I suspect that these numbers are not necessary.

The second, int2, is the "category". This is [00] for the initial-meeting greeting (which always mentions the name of the character), and [01] for the subsequent-meetings greeting. All other values are taken from int11, described below.

Then come the two "conversation" strings. Either or both can be zero length. Each conversation string is stored as int stringlength, then the ascii characters, with no terminator, like "[04]Fred". So a zero length one is [00]. In my examples, I put [LL] to mean "I'm too lazy to calculate this".

In these strings, the \n character can be used on its own within a conversation string to break the line. This is most often seen as \n\n for a blank line. The $ character will be replaced by the player name (but only at display time: it counts as a single character for the string length int, of course!)

The exchange is followed by seven ints (which I've numbered int5 to int11), then four "data" strings. Unlike the conversation strings, these have no length int, and are terminated by \r\n.

The number in int5 is a quest number to test, or zero. Then int6 stores the quest stage to test against, or zero. If both are non-zero, then the exchange will only be available if the quest is at that stage. There is no case where only one of these ints is zero.

Int7 is unknown. It is only set in the files listed below. Other than zero, no value appears in more than one file. No file contains more than two values. Most values are used once only, but 14, 15, and 27 are used up to three times. Higher file numbers contain higher int7 numbers. The numbers do not usually correspond to any exchange id in the file.

Int8 is unknown. It is set in the same files as int7, except for 63 or 76. Values appear across multiple files, and multiple times in the same file. The numbers always correspond to an existing exchange id in the file, but this may be coincidence, as they are all low numbers.

Int9 and int10 are always zero.

Int11 is the next conversational category to display options for. -1 sends the player back to the initial greeting, and is only used in exchanges where the NPC string is empty. If int11 points to a category that does not exist, then the conversation terminates. However, in this case, there is always an "empty" exchange (see below) with the same exchange ID as the non-existent category ID.

The first data string, str3, is before the first \r\n, and is an optional item name, which I believe is the name of an item that needs to be in the PC inventory in order for the exchange to be offered as an option in the list. This is typically used when removing items:

With the rename_item command, this seems occasionally used as another way of tracking quest statuses, if another variable is needed instead of the the normal quest level mechanism.

As shown above, any script for the exchange is placed in the second data string, str3, before the next \r\n. The script is run before the NPC response is displayed, so the following exchange is displayed in a sensible order:

The next two data strings, str5 and str6, are always empty. Possibly one is for a script that runs after the NPC's text displays?

Since the only required part of the exchange is the incrementing id, you can and do see 'empty' exchanges like this. One of these exists for every conversational category that terminates a conversation, with its category id set to 0 and its exchange id set to the category id. I don't know why:



So here's a full conversation tree:



That becomes:


Now, I'm sure you can see scope for immediate optimisation there - all four "[Back]" blocks can be combined into one by giving them the same category ID, for a start. And those empty exchanges, same thing.

Except, this is never done. I don't know why: I assume the software they used to generate these files didn't consider the possibility of reusing exchanges for multiple responses.

But what that means is that there's plenty of space to fit in another conversation option or two into most conversation files.

OK, think I got all the procrastinatory marrow out of this bone that I can be bothered to get. Dropping it now.

[Edit: filled in the two unknown ints, so now we know how the trees work.]

Found this in another thread:


Well, dang. I was considering making a little something to allow conversations to be modded, but while that might've been fun, I feel that it's much more important to respect BW's restrictions on his own work.

I'll happily delete any/all my above posts that're against those restrictions, and am happy for them to be deleted by admins, too.

Apart from SpottedShroom and myself I find it disappointing that there hasn't been more content created and released with this editor. Nobody working on something?

Well, I posted that on November 1 last year, so it seems only fair that I should actually follow through on my promise before we get to a whole year's delay. The suggestions from both folks have now been integrated into the scripting docs on my site.

In other news: hello, everyone! I've not been entirely absent over the past year - I've been popping in every month or two to catch up on threads - but I have definitely been letting these utilities fall by the wayside. I had ended up maintaining a utility for Minecraft which has pretty much sucked up all the time that I'd have been spending on these utilities. I've been meaning to hop back in here, though, and some recent activity in the Book II editor thread seems to have spurred me into action.

In addition to integrating the extra scripting notes (and hopefully making a few more changes before I push out a new release of the utilities) there's a couple of other changes. One is that I've set up a Sourceforge project here: https://sourceforge.net/projects/eschalonutils/ - The "main" website for the utilities will remain as it always has (linky), but Sourceforge is where the actual file releases will be hosted, and it might be useful to some folks because of its RSS integration and all that (this, of course, assuming that I start maintaining this thing a little better than I have over the past year).

I've also moved the code repository from being a private, locally-hosted CVS thing on my own fileserver, to git. The sourcecode was always distributed inside the .tgz and .zip distributions, but now you can also find it at github: https://github.com/apocalyptech/eschalon_utils - I'm also pushing any changes to Sourceforge's repository, though I consider github to the the origin. One upswing of that is that anyone could have access to unreleased code if you want, including a pretty nifty script editor component which, it turns out, hadn't actually made it into a real release yet (whoops!)

So yeah, that's that. Hopefully I should have another release of this stuff out within a week or so!

Good to see you're still making improvements to the editor. Please allow me to make an unreasonable feature request. I'd love to see an API into the map interface, so that I could write code like this:

Heh, yeah, I'd love to see that stuff cleaned up, especially after having used the incredible pymclevel, a Python library for editing Minecraft worlds. I also noticed someone's thread asking for an "infinite dungeon generator," and got to thinking that it might be pretty fun to try and whip something up to do so.

You should be able to at least do some of what you want... One of the bigger problems would be having to deal with the coordinate system yourself, which isn't very intuitive or easy to work with. Your example code can be more-or-less replicated like so, btw:


Apart from clearly not really being well-API'd, the big flaw there is that the coordinates don't really work like that, and you end up with a map that looks like:

Not exactly ideal. A routine to draw a "straight" wall might look like:


So yeah, there's that for now, at least. Some digging through my probably-labyrinthine code might help as well.

I will see what I can do about cleaning all that up, providing some actually-useful API calls for walls and the like, integrating the smartdraw stuff, and providing actual documentation for it... That'll probably take a bit, though, and I probably won't target that until after 0.8.0.

I just discovered this editor, great work !
Now i have two questions:

1. What does the "North Exit", "East Exit"...inside the "Global Map Properties" actually do ?

2. I had the idea to just create a very basic quest (go there and kill something) but cant see where to start. I know i need to place a person to talk with (to get the quest) but after i placed this person i dont know what to do, how can i add a quest to that person so i can talk to him and accept the quest and see it inside my quest-log ?

Oh, i already looked here but found no answers to my questions:
http://apocalyptech.com/eschalon/b1scripting.php

The various exit properties decide what map you will arrive on when you walk off the matching edge of the screen.


Unfortunately at the moment it is not possible to edit the npc dialog, in my mods I have dealt with adding quests to some degree using signs, letters etc to give instuctions using eith the message () or condition () functions.

Thanks !

I hope this will be possible at some point...
Do you know what the .ent files do ?
There seems to be one for every map lile 35.ent.

Oh wait, that just the entities right ?
But what the locaton of the quest scripts then ?

Oh, and if i want to create my own map or dungeon, where would i start ?
Would i create a "new" global map, save it with some number and then just add a new map exit to it ?
So do i only need to edit a "Global Map" or is there something to do in "Savegame map file" mode ?
I hope you can understand what i mean with that, as you know there are two ways to load a map (from a Savegame & from the Data folder).

Thanks !

You can create either a global or save game map, since I made my mod for Book I as a global one, but once I moved on to Book II I was forced to do it as a save game one due to changed in the game setup but I discovered that's actualy better as when creating a save game map you can define your own items, weapons, keys etc.

You will then need to edit one of the existing maps to give access to your new maps, either by setting one of the Exit's in the map properties or setting up a Map Link object on a teleporter, staircase, cave enterance etc.

Thanks again.
Is there a special way i need to name my new map ?
Or can i just use "test" as name for example ?

And if i create a new "Save Game Map", can i just put it in the Data folder and use it as Global map ?