Cyber Knight translation

[NightWolve]

I would suggest the use of a simple emulator like YAME for what you seem to be describing, wanting to load a state file and get to test new changes within the ROM. It's simple, has a full GUI and is compatible with the game having just checked it.

https://www.pcengine-fx.com/downloads/yame038.zip

Setup Instructions:

1) Launch it.
2) Goto -> "Settings" -> "Advance Settings" -> click "Input Tab".
3) Under "Save" press F1, Under Load, press F5 : Now F1 is a quick key to save a state file, F5 to load it.
4) Configure the rest of the buttons to the keyboard as you prefer.
5) Load the ROM under the File menu "Open" command OR drag'n'drop the ROM file on its window, simple as you'd expect.

Now, here's the thing, when you change the ROM file, but you use the state file to resume the game exactly where you were, you have to go somewhere in the game to force the loading of new ROM data otherwise you may not see your changes. The other way is to load the state file, save your game (if possible) via the game's menu, reset the game, and load it again the same way. That'll force a new loading of ROM data into the system RAM, etc. But, I guess you already knew that and want to avoid it. So yeah, the only other way would be to move around in the game to where a new event is loaded or a new level is, etc.

EDIT: This emulator does not use a checksum system for dealing with state files. The state file will simply be named the same as the ROM but with a ".sta" extension, e.g. "Cyber Knight (J).sta". YAME is perfect for this job, for you, etc. It was very handy when it came to hacking Xak III/Ys IV ages ago for its simplicity and quickness when it comes to insert/test-in-game/insert/test-in-game/etc. You're going through some unneeded headaches/tediousness using a command-line Mednafen, and unless it's time to use its debugger for something advanced, you can do better in the meantime, so I'd stick with YAME for most things (once you get the hang of it, which you should no problem).

[megatron-uk]

That's great - I managed to find a binary of yame that works on Linux (no recent versions unfortunately - so no sound support, but that's not important). I'll give that a go this evening.

Cheers,
John

[NightWolve]

Oh, you're a Linux guy, huh ? Hmmmm. Well, I have no idea what the interface is like on a Linux build for it, didn't even know the developer made one. Would've been really convenient for hacking to work on a Windows box for you, but oh well. You've managed fine so far anyhow.

[cabbage]

Anyway, what I was wondering is there any way Mednafen can be used to load a state file if the underlying rom file has changed? It's a pain adjusting the rom, reinjecting modified text and then playing right through about 10 or 20 battles to get to the point where Vynd says his text again. I believe the states saved by Mednafen only load when the rom has the same checksum, which it obviously doesn't when I change the data....

Go into the save-state directory (mcs) and remove the trailing checksum from the save filename.
e.g. something like CyberKnight.214a55e19aed90a0095ca0d65894b7d3.mc0 can be renamed to simply CyberKnight.mc0 and it should load even without a checksum match

[megatron-uk]

Thanks to both of you. Both methods work, but I'm still stuck with the problem of refreshing the rom file from disk. Savestates load fine now (thanks for the checksum tip, Cabbage), but at that point, the rom data in memory (checked with mednafen debugger) still reflects the pre-modified file. I've tried loading the status screen, finding a battle to fight, viewing another on-screen text dialogue etc, short of having to exit and leave the main playfield (which is the whole point of using a save state to bypass a dozen or so battles) I can't seem to find anyway past this hurdle.

[Edit] Still can't find a way of refreshing the rom data when using either emulator like this, but on this occaision just using the debugger to to alter one or two bytes was sufficient. There was a sequence of bytes embedded in the text sequence on that screen:

Code Select Expand

"08", "13", "3a", "6a", "00"

Now 08 is 'pause until button pressed' and 00 is 'end of string'. I don't know what the 13/3a/6a sequence is, but it appeared to jump somewhere else (way, way off, into text for the next world - the 'Trader' quests), rather than printing the 'drive coil found' string which was the next thing to show. This seems to be in the original rom too, so I'm not sure what it's actually doing.

For now, I've replaced those 'jump bytes' (for want of a better term) with my 'no-op' codes - 0x5c (actually, character shift, but they have the effect of padding without printing anything on the screen), and the result is as shown in the image below!

[Dicer]

That's great - I managed to find a binary of yame that works on Linux (no recent versions unfortunately - so no sound support, but that's not important). I'll give that a go this evening.

Cheers,
John

Magic Engine runs under Wine in Linux quite perfectly, just a heads up there.

[megatron-uk]

As you're probably aware, I'm using the SNES version of Cyber Knight to help in the translation. I have a dump of the English script so that's the basis of the text, it's helped a lot.

I've now played and translated everything up to and including the defeat of the dome on the southern continent and am now back talking with the president (and receiving his deepest thanks for dealing with the robot threat).

In the SNES version the dialogue goes something like this:

"Our deepest thanks, the planet is now free"
"You want to go back to earth, yes?"
"I'm afraid we cannot help with our technology"
"The Traders mentioned a race called the Seeclan with advanced technology, maybe they can help"
"Ask details from the Traders. You can find them on Worldring-3"

At which point I presume you then use the rebuilt jumpdrive and move to the Worldring system.

In our PCE version the President says basically the same thing but his last sentence is different:

"Our deepest thanks, the planet is now free"
"You want to go back to earth, yes?"
"I'm afraid we cannot help with our technology"
"The Traders mentioned a race called the Seeclan with advanced technology, maybe they can help"
"『‥‥‥ただ‥‥‥さいきん れんらくが とだえているそうですが‥‥‥』"

I think the last line is something along the lines of 'we had regular contact with them, but it was suddenly cut-off recently'. - that's the last thing he says - no mention of the Worldring system. Hmm, now how do I know I'm supposed to get to the Worldring system?

[TurboXray]

In mednafen, if you want to reload the rom, then you'll have to do it through the debugger alt+3 tab/window. Go to that, change the window to 'cpu physical'. Use the L option to load a file into start/end part of external address range. Just make sure the rom is raw (headerless), else it'll load the header as well.

 Be careful, as some games load part of the text string from rom, into ram. So this will mess you up for savestates.

[megatron-uk]

Ah, so that's how to do it! Nice one Tom. That will be very helpful, thanks!

[whisper2053]

I feel incredibly dumb right now.

Python is installed, I have your archive from github, and I acquired the proper rom in hopes of being able to perhaps test this out. For the life of me, I cannot figure out how the heck to get Python to execute the appropriate extract and inject .py's and it's driving me crazy.

Sigh.

Had to vent, sorry

[megatron-uk]

What happens when you run:

Code Select Expand

python injectScript.py -i "Cyber Knight (J).pce" -o "Cyber Knight (E).pce" -d ./patches/
Sorry for lack of updates recently. House finally sold, everything is packed up or in storage and we're waiting to move into our new home (end of June, most likely) .

[whisper2053]

Granted, I'm almost certain I am doing something wrong, as I have never used Python before.

[megatron-uk]

Don't type 'python' or double click the python icon - it starts an interactive shell; you're running python code at that point. You just need a Windows command prompt open.

Open a command prompt and paste in the full line that I wrote, above.

Something like (presuming you've created a directory called 'cyber' will all the downloaded files and folders in). As long as Python.exe is in your path, you should be able to do something like this:

Code Select Expand

c:\> cd cyber
c:\cyber>
c:\cyber> python injectScript.py -i "Cyber Knight (J).pce" -o "Cyber Knight (E).pce" -d ./patches/
I do intend to distribute an ips patch once the translation is complete, but for now, having access to the source of the translation is easier for those with a programming background who may want to help out or try their own script.

[whisper2053]

Have there been any further updates on this project?

Also, I finally figured out what my previous issue was (ie, I did *not* have Python.exe in my path, and that is fixed now), but I am running into a new issue currently. When I attempt to run the string you provided above, I receive this error:



Hopefully some light can be shed on this

[megatron-uk]

I've done a little more - I have some changes that take the translation up to the end of the first world, but these are not committed to GIt yet. I haven't made as much progress as I would have liked as I'm constantly hitting free-space issues in the script.... there is some free space in most text segments, but it seems random whether they can be used or not (there's no lookup table as such - each string is simply delimited from the next one and the print routines loop over the delimiters until they get to the correct string).

In terms of the error you're getting, I think you must be using Python3 - the 'print' call changed from Python1/Python2, which used the directive:
print "STRING"

to be a proper function called like this in Python3:
print("STRING")

Fortunately the latter format is backwards compatible with Python2 - so I could go back through the code and change it and it would be compatible with all Python versions.

[CrackTiger]

Has this project progressed any further?

[megatron-uk]

Sorry folks, real life has gotten in the way. I haven't had the time to touch this for probably a year now - a promotion at work to managing a team (and all the paperwork that comes with!) and more responsibilities at home mean I just don't have the time to devote to it.

[esteban]

Sorry folks, real life has gotten in the way. I haven't had the time to touch this for probably a year now - a promotion at work to managing a team (and all the paperwork that comes with!) and more responsibilities at home mean I just don't have the time to devote to it.

I hear you



Thanks for the update.

[megatron-uk]

Hello, me again

So, I've decided to have a look at Cyber Knight again, after a break of over 2 years. Now's the time to do it, as things are probably going to change quite dramatically in my life in the next 6 months or so and I imagine I'll not have any spare time for the next ... ooh, 18 years or more!

So, where I got to with the translation was a point where the English text simply won't fit in to the space available. I had to start overflowing in to areas of the ROM that appear to be blank, although this gave me some extra space for NPC text for the first world, it broke the scrolling intro and I got random rubbish after finishing the first world, meaning I couldn't progress.

I really need the assistance of someone with more PCE programming knowledge who can help by either changing the text routines to point to blocks outside the main code section, or expand the ROM size, and, again move the text their. There just isn't the space available to replace the bulk of the text without overflowing in to the code sections.

Just as a reminder, Cyber Knight uses a fairly crude text block system, which makes it relatively easy to insert: the text is stored in several positions throughout the ROM, interspersed by code and graphics data. The text strings simply run on from each other and are seperated by a given byte sequence, the text display routine just counts over these text blocks until the right number of skipped sequences have been counted, then displays the next sequence. It makes it easy to modify the text (and its uncompressed, which is a big bonus for me). But expanding beyond the availabvle space, or moving it altogether is out of my league.

So, here's a request for help - would anyone be willing to assist in this particular part of the translation, i.e. changing the text routine, and/or expanding the ROM to include enough space to store the english text?

It's a shame for more of us not to get to play it, it really is a great little hucard rpg/strategy game.

[NightWolve]

Hey megatron, forgot about ya! You were the best status update provider I had seen, well, before elmer and SamIAm came along, that is. ;) Try talking to Bonknuts for help, elmer's too busy with his projects. Too bad you can't do it yourself, anyway, all the best!

[elmer]

I really need the assistance of someone with more PCE programming knowledge who can help by either changing the text routines to point to blocks outside the main code section, or expand the ROM size, and, again move the text their. There just isn't the space available to replace the bulk of the text without overflowing in to the code sections.

Just as a reminder, Cyber Knight uses a fairly crude text block system, which makes it relatively easy to insert: the text is stored in several positions throughout the ROM, interspersed by code and graphics data. The text strings simply run on from each other and are seperated by a given byte sequence, the text display routine just counts over these text blocks until the right number of skipped sequences have been counted, then displays the next sequence. It makes it easy to modify the text (and its uncompressed, which is a big bonus for me). But expanding beyond the availabvle space, or moving it altogether is out of my league.

OK, a quick look at the game's scripting system shows that the data in the ROM uses a simple overall structure.

The scripts/data is broken up into 16KB blocks that are mapped into the $4000-$7FFF region.

Bank $0A/$0B, ROM offset $14000-$17FFF
Bank $0C/$0D, ROM offset $18000-$1BFFF
Bank $0E/$0F, ROM offset $1C000-$1FFFF
Bank $14/$15, ROM offset $28000-$2BFFF

Each block starts with 1 byte bank number (presumably just for reference)

Then there's a table of 2-byte pointers to each of the individual asset chunks within the block (a chunk is either script or graphics).

Those pointers are always going to be in the range $4000-$7FFF.

There's a table in bank $01 (mapped into $c000-$dfff) that the code uses to locate each of the script chunks, it's split into 2 parts.

The 1st part gives the bank number to map into $4000-$7fff, and the 2nd part gives the low byte of the address of the table of asset chunk pointers to load the actual pointer from to get to one of your "text blocks".

A number of those text blocks are contiguous in memory, so if you're just looking at the raw data in the ROM, then you're probably losing track of what the game thinks of as the actual start-end of the different individual asset blocks.

With this info, you should be able to expand the game's ROM and relocate any of the text blocks into the extra memory and so avoid the problems that you're having.

The 1st table of bank numbers is at $C939 (ROM $02939), and the 2nd table of asset pointer offsets is at $C95A (ROM $0295A).

The contents of those tables are ...

; rom $02939 : script bank

$c939 0e 0e 0e 0e 0e 0e 0c 0e
$c941 0e 00 00 00 00 00 00 00
$c949 14 0a 0a 0a 0a 0a 0c 0c
$c951 0c 0c 0c 0c 0c 0c 0c 0c
$c959 0c

; rom $0295a : script offset

$c95a 01 03 05 07 09 0b 17 0d
$c962 0f 00 00 00 00 00 00 00
$c96a 83 01 03 05 07 09 01 03
$c972 05 07 09 0b 0d 0f 11 13
$c97a 15


Does that make any sense?

[megatron-uk]

That does indeed explain why some of the expanded text seems to have broken other, unrelated parts of the game.

I think I've got my head around most of what you've explained (and thank you very much for taking the time to do so!). Those 4 text banks you've identified overlap with the text sections I have extracted and am injecting back in, so that makes sense why the bulk of what I'm doing works, but also why it starts to go awry later one.

I have a couple of clarifications though:

- In the 16kb text block 'banks' (0x0a/0x0b, 0x14/0x15 etc), In the table of 2-byte asset pointers immediately following the text bank number, how do I know how many of those such pointers there are before the assets begin?

- In the main bank 0x01, in the second table, what are those asset pointer offsets doing? I can see that when bank 0x14 is loaded for example, the corresponding offset is 0x83. But what does that actually result in?

Many thanks for the information so far, I think this will be an incredible step forward. I was at the point of contemplating going back to fixed-length string insertions, which, while it would have worked, would have resulted in some awful dialogue compared to the original Japanese.

[megatron-uk]

Okay, I think I can answer the first question myself now, having looked at one of the asset banks in more detail.

Here's the start of 0x0C bank:

ROM position 0x18000-0x1BFFF

0C 19 40 62 44 61 4C 47 51 27 59 16 5C 4B 63 17 6E A2 6E D5 71 DB 75 CE 78 00 .....

So, if I follow your advice, the first byte, 0x0C is the text bank number. Then there is a single byte 0x19 which is 25 in decimal. Count 25 bytes, take off 1 for the 'size' and then that's the number of pointers in the table. That gives the following:

Bank: 0x0C
Size: 0x19
Pointers: 0x4062, 0x4461, 0x4C47, 0x5127, 0x5916, 0x5C4B, 0x6317, 0x6EA2, 0x6ED5, 0x71DB, 0x75CE, 0x7800

That seems to work for the other asset banks, too, with asset bank 0x14 the biggest, with a 147 byte table.

So, (now the gears are turning in my mind...) does that mean that when bank 0x0E is loaded from the first entry in the main bank 0x01, it jumps to pointer 1 (offset 0x01 from the offset table) in the table? Likewise, the last load entry for table 0x0C jumps to position 0x15 in the header table?

Have I got that right?

I guess that once it jumps to pointer position 0x15 (which will point to a block of text at 0x5FFF, say), it will then follow the logic from before, of looping over the script fragments in that block at 0x5FFF, skipping end-of-string bytes until it finds the 93rd one (for example), then displays it?

[megatron-uk]

To clarify the number of pointers at the start of an asset bank:

Second byte = 0x19 = 25 = 24/2 = 12x 2-byte pointers

I think that's correct.

[megatron-uk]

Okay, I've written up a summary of the asset banks so far.

Code Select Expand

==============================================

Master Bank $01, always loaded at 0xC000-0xDFFF

Table A, ROM offset 0x02939, type = Script Text Bank pointers
This is a table of which Text Bank to load at any point.

Start byte position = 10553

Data:
0e 0e 0e 0e 0e 0e 0c 0e
0e 00 00 00 00 00 00 00
14 0a 0a 0a 0a 0a 0c 0c
0c 0c 0c 0c 0c 0c 0c 0c
0c

There are 33 bank load addresses, many of which, as shown, are duplicates.

---

Table B, ROM offset 0x0295a, type = Script Offset pointers.
This is a table of offsets into the pointer tables at the start of each text bank.

Start byte position = 10586

Data:
01 03 05 07 09 0b 17 0d
0f 00 00 00 00 00 00 00
83 01 03 05 07 09 01 03
05 07 09 0b 0d 0f 11 13
15

There are 33 asset chunk pointer offset addresses, one for each of the bank load entries in Table A.

=============================================

The below text banks are always loaded at 0x4000-0x7FFF

=============================================

Text Bank $0A/$0B, ROM offset $14000-$17FFF
First byte = Bank number = 0x0A
Second byte = 0x0B = 11 = 10/2 = 5x 2-byte pointers

Pointer Data
-------------
0A 0B 40 FD 54 6F 6A 29
6C C1 75 00

Pointer Table
----------------
Position / Value
0x00 = 0A = Bank ID
0x01 = 0B = Table size
0x02 = 40 FD - 0x4000 + 0x14000 = 0x140FD
0x03 = 54 6F - 0x4000 + 0x14000 = 0x1546F
0x04 = 6A 29 - 0x4000 + 0x14000 = 0x16A29
0x05 = 6C C1 - 0x4000 + 0x14000 = 0x16CC1
0x06 = 75 00 - 0x4000 + 0x14000 = 0x17500

=============================================

Text Bank $0C/$0D, ROM offset $18000-$1BFFF
First byte = Bank number = 0x0C
Second byte = 0x19 = 25 = 24/2 = 12x 2-byte pointers

Pointer Data
-------------
0C 19 40 62 44 61 4C 47
51 27 59 16 5C 4B 63 17
6E A2 6E D5 71 DB 75 CE
78 00

Pointer Table
----------------
Position / Value
0x00 = 0C = Bank ID
0x01 = 19 = Table size
0x02 = 40 62 - 0x4000 + 0x18000 = 0x18062
0x03 = 44 61 - 0x4000 + 0x18000 = 0x18461
0x04 = 4C 47 - 0x4000 + 0x18000 = 0x18C47
0x05 = 51 27 - 0x4000 + 0x18000 = 0x19127
0x06 = 59 16 - 0x4000 + 0x18000 = 0x19916
0x07 = 5C 4B - 0x4000 + 0x18000 = 0x19C4B
0x08 = 63 17 - 0x4000 + 0x18000 = 0x1A317
0x09 = 6E A2 - 0x4000 + 0x18000 = 0x1AEA2
0x0A = 6E D5 - 0x4000 + 0x18000 = 0x1AED5
0x0B = 71 DB - 0x4000 + 0x18000 = 0x1B1DB
0x0C = 75 CE - 0x4000 + 0x18000 = 0x1B5CE
0x0D = 78 00 - 0x4000 + 0x18000 = 0x1B800

=============================================

Text Bank $0E/$0F, ROM offset $1C000-$1FFFF
First byte = Bank number = 0x0E
Second byte = 0x11 = 17 bytes = 16/2 = 8x 2-byte pointers

Pointer Data
-------------
0E 11 40 4C 4B ED 4E 73
53 B2 55 86 5C FF 6B EC
70 00

Pointer Table
----------------
Position / Value
0x01 = 0E = Bank ID
0x02 = 11 = Table size
0x03 = 40 4C - 0x4000 + 0x1C000 = 0x1C04C
0x04 = 4B ED - 0x4000 + 0x1C000 = 0x1CBED
0x05 = 4E 73 - 0x4000 + 0x1C000 = 0x1CE73
0x06 = 53 B2 - 0x4000 + 0x1C000 = 0x1D3B2
0x07 = 55 86 - 0x4000 + 0x1C000 = 0x1D586
0x08 = 5C FF - 0x4000 + 0x1C000 = 0x1DCFF
0x09 = 6B EC - 0x4000 + 0x1C000 = 0x1EBEC
0x0A = 70 00 - 0x4000 + 0x1C000 = 0x1F000

=============================================

Text Bank $14/$15, ROM offset $28000-$2BFFF
First byte = Bank number = 0x14
Second byte = 0x93 = 147 bytes = 146/2 = 73x 2-byte pointers <- INCORRECT, only 66 pointers in this table.

Pointer Data
-------------
14 93 6D 95 6D A4 6D 36
6E 65 6E B7 6E 95 6D 0E
6F 95 6D 3D 6F 5E 70 95
6D 84 70 95 6D 93 6D 45
71 71 71 02 72 37 72 B2
72 95 6D 0E 73 DF 73 95
6D 20 74 81 74 AA 74 BD
74 D4 74 93 6D 93 6D 42
75 95 6D 74 75 B4 75 95
6D F7 75 49 76 95 6D 6F
76 F4 76 B4 77 CF 77 CF
77 95 6D 93 6D 93 6D 93
6D 93 6D 93 6D 93 6D 93
6D 93 6D 93 6D 93 6D DA
77 95 6D FA 77 6C 78 C1
78 E4 78 0C 79 93 6D 3D
79 15 6A 85 40 00

Pointer Table
----------------
Position / Value
0x01 = 14 = Bank ID
0x02 = 93 = Table size <- INCORRECT

TODO: The table size byte (0x93 == 147) doesn't appear to correlate to the actual number of entries (133 - 1 / 2 = 66) in the table for bank 0x0E, as there are not 73 entries in the table.
One problem I've found is that the second byte == pointer table size assumption doesn't hold true for bank 0x14/0x15 - it should have a 147 byte pointer table == 76 pointers, but it only appears to have 66 pointers if we assume they all have to be within 0x4000-0x7FFF. Something odd happening with that one.

[elmer]

Okay, I've written up a summary of the asset banks so far.

Code Select Expand

==============================================

Text Bank $14/$15, ROM offset $28000-$2BFFF
First byte = Bank number = 0x14
Second byte = 0x93 = 147 bytes = 146/2 = 73x 2-byte pointers <- INCORRECT, only 66 pointers in this table.

Pointer Data
-------------
14 93 6D 95 6D A4 6D 36
6E 65 6E B7 6E 95 6D 0E
6F 95 6D 3D 6F 5E 70 95
6D 84 70 95 6D 93 6D 45
71 71 71 02 72 37 72 B2
72 95 6D 0E 73 DF 73 95
6D 20 74 81 74 AA 74 BD
74 D4 74 93 6D 93 6D 42
75 95 6D 74 75 B4 75 95
6D F7 75 49 76 95 6D 6F
76 F4 76 B4 77 CF 77 CF
77 95 6D 93 6D 93 6D 93
6D 93 6D 93 6D 93 6D 93
6D 93 6D 93 6D 93 6D DA
77 95 6D FA 77 6C 78 C1
78 E4 78 0C 79 93 6D 3D
79 15 6A 85 40 00

One problem I've found is that the second byte == pointer table size assumption doesn't hold true for bank 0x14/0x15 - it should have a 147 byte pointer table == 76 pointers, but it only appears to have 66 pointers if we assume they all have to be within 0x4000-0x7FFF. Something odd happening with that one.

You've made a logical-deduction that makes sense from a programmer's point-of-view, but it doesn't actually match reality. 

You're making the mistake of assuming that the 1st asset chunk in the 16KB block is actually always going to be the 1st entry in the table (I made that same mistake when I was doing the Zeroigar translation).

Since the table is a list of pointers, and the game will just refer to assets by block number & index number, then it's OK for a data block to have things stored out-of-order.

That's the case with bank $14 ... if you look, the last pointer in the table (at $4083) is the pointer to that script chunk in there that starts at $4085.

So the script chunk is located 1st in the 16KB block, but it's asset-index is $83, and all the other graphics chunks have lower asset-index values.

So to actually find out how many assets are in a 16KB block, you've got to read all of the 2-byte pointers, as 2-byte pointers, from the start of the table onwards, keeping track of the lowest value that you see, and then stop reading when you hit the point that the pointer to the table is the same as the lowest asset pointer.

Or you can just do it by-eye since there are so few chunks that you'll need to modify.

[elmer]

So, (now the gears are turning in my mind...) does that mean that when bank 0x0E is loaded from the first entry in the main bank 0x01, it jumps to pointer 1 (offset 0x01 from the offset table) in the table? Likewise, the last load entry for table 0x0C jumps to position 0x15 in the header table?

Have I got that right?

I guess that once it jumps to pointer position 0x15 (which will point to a block of text at 0x5FFF, say), it will then follow the logic from before, of looping over the script fragments in that block at 0x5FFF, skipping end-of-string bytes until it finds the 93rd one (for example), then displays it?

It's all just a way for the original game programmers to be able to move individual asset-chunks around inside a sort of pseudo file-system in the ROM so that they can pack each ROM bank as-full-as-possible and just refer to assets as a 2-byte bank+index.

Remember, ROM is (was) expensive, and the challenge of packing things into cartridge space was one of the biggest problems of that era of game programming.

The tables at $02939 & $0295a for the script bank and offset look like they're just for the text blocks.

There could easily be some other tables somewhere for graphics blocks or other assets, or maybe the original developers just used a list of equates/#defines for those in their source code.

The big thing is ... there's now nothing stopping you from creating new 16KB blocks of text assets in an expanded ROM for the translation, and then moving some/all of the English text asset-chunks into the new blocks.

You'd only need to change the tables at $02939 & $0295a to point to your new asset chunks.

You can do the same when it comes to changing any of the graphics that you want to (like the lower-case descenders on that font).

[megatron-uk]

I think I've got it, but I want to run through the pointer system a few times so that I'm clear. I think I've got where asset index $83 is coming from, and what it relates to, but want to be sure before asking more inane questions 

I'll play around with a few examples this evening first.

[elmer]

I think I've got it, but I want to run through the pointer system a few times so that I'm clear. I think I've got where asset index $83 is coming from, and what it relates to, but want to be sure before asking more inane questions 

Things become clearer if you just reformat the data so that it's more like what a programmer would write ...

ROM     Idx   Pointer
----------------------
$4000         db  $14
$4001   $01   dw  $6D93
$4003   $03   dw  $6D95
$4005   $05   dw  $6DA4
$4007   $07   dw  $6E36
$4009   $09   dw  $6E65
$400b   $0b   dw  $6EB7
$400d   $0d   dw  $6D95
$400f   $0f   dw  $6F0E
$4011   $11   dw  $6D95
$4013   $13   dw  $6F3D
$4015   $15   dw  $705E
$4017   $17   dw  $6D95
$4019   $19   dw  $7084
$401b   $1b   dw  $6D95
$401d   $1d   dw  $6D93
$401f   $1f   dw  $7145
$4021   $21   dw  $7171
$4023   $23   dw  $7202
$4025   $25   dw  $7237
$4027   $27   dw  $72B2
$4029   $29   dw  $6D95
$402b   $2b   dw  $730E
$402d   $2d   dw  $73DF
$402f   $2f   dw  $6D95
$4031   $31   dw  $7420
$4033   $33   dw  $7481
$4035   $35   dw  $74AA
$4037   $37   dw  $74BD
$4039   $39   dw  $74D4
$403b   $3b   dw  $6D93
$403d   $3d   dw  $6D93
$403f   $3f   dw  $7542
$4041   $41   dw  $6D95
$4043   $43   dw  $7574
$4045   $45   dw  $75B4
$4047   $47   dw  $6D95
$4049   $49   dw  $75F7
$404b   $4b   dw  $7649
$404d   $4d   dw  $6D95
$404f   $4f   dw  $766F
$4051   $51   dw  $76F4
$4053   $53   dw  $77B4
$4055   $55   dw  $77CF
$4057   $57   dw  $77CF
$4059   $59   dw  $6D95
$405b   $5b   dw  $6D93
$405d   $5d   dw  $6D93
$405f   $5f   dw  $6D93
$4061   $61   dw  $6D93
$4063   $63   dw  $6D93
$4065   $65   dw  $6D93
$4067   $67   dw  $6D93
$4069   $69   dw  $6D93
$406b   $6b   dw  $6D93
$406d   $6d   dw  $6D93
$406f   $6f   dw  $77DA
$4071   $71   dw  $6D95
$4073   $73   dw  $77FA
$4075   $75   dw  $786C
$4077   $77   dw  $78C1
$4079   $79   dw  $78E4
$407b   $7b   dw  $790C
$407d   $7d   dw  $6D93
$407f   $7f   dw  $793D
$4081   $81   dw  $6A15
$4083   $83   dw  $4085


In particular, note the multiple table entries that refer to the same asset-chunk at $6D93 & $6D95.

That's some programming-trick going on, possibly for level-loading or something like that where you want to use the same graphics for the main character in the game.

[elmer]

BTW ... there's no substitute for running the game in Mednafen when it comes to understanding what's going on (if you understand assembly language).

I'd recommend using my custom-built version that's linked in one of the other threads because the new larger fonts make it so much easier to read.

If you run the game in there, then here are the locations of some useful routines (all logical game addresses, and not ROM offsets) ...

$f1e0 - code that maps a 16KB data block into $4000-$7fff.
$f1ce - code that takes a Bank & Asset-Index and maps in the bank and returns a pointer to the asset.

$cd32 - code that takes a script number, and uses the tables at $c939 & $c95a to map in the correct data bank and then load the pointer to the script asset into $30/$31.

[megatron-uk]

I know why I'm confused.

The pointer index table is little endian. I was reading it as big endian.

I couldn't work out where you were getting the pointer $4085 from, since my last pointer was $4000. But of course it's not, and the second byte isn't a counter at all.

SO where I thought the 0x0A table (which is 0A 0B 40 FD 54 6F 6A 29 6C C1 75 00)...

0x0A (asset bank id)
0x0B (table pointer size)
40 FD (pointer)
54 6F (pointer)
... and so on.

it's actually (yes, you can slap me now):
0x0A (asset bank id)
40 0B (pointer)
54 FD (pointer)
... etc.

I'll go sit in the corner now. 

[elmer]

I'll go sit in the corner now. 

Hahaha ... nope, now you'll be too busy working on it all to take time off and sit in the corner! 

It's a very easy mistake to make, don't worry about it.

Once you've got the translation fixed up, it'll all be about the presentation.

Please, please, please consider customizing the font a bit. 

Lower-case descenders that appear completely above the baseline just make my eyes bleed! 

If you can't afford the extra lines to drop those descenders, then you might want to consider changing the lower-case part of the font into small-caps instead (which also gives you the excuse to put crossbars on the "i" and "l" so that they don't look so thin and out-of-place next to the other fixed-width letters).

Or not ... just an IMHO. 

[elmer]

Once you've got the translation fixed up, it'll all be about the presentation.

Please, please, please consider customizing the font a bit. 

Lower-case descenders that appear completely above the baseline just make my eyes bleed! 

OK, a quick look at the game (I'd never seen if before yesterday), shows that all the text is done as 8x8 background tiles, and not dynamically drawn into sprites.

That means no VWF, but, on the positive side, every piece of text, and box, that I've seen is spaced to allow for 16-pixel-high Kanji.

So once the game is 100% translated, and the Japanese font is no longer needed, it should be possible to modify both the font and the display code so that all of the english text is drawn as 2 8x8 tiles ... allowing for real descenders, and a slightly taller font, all without screwing up the overall look.

Since there's also the whole Kanji font bank in there that's permanently loaded as well, that space could potentially be used to make custom bitmaps for some of the game's options, making those look even prettier.

Oh, and just for giggles, I checked, and it looks like there is a permanent "black" in the text palette, so it would be possible to add a drop-shadow to the text, even though it doesn't strictly need one!

Finally ... there's a ton of unused space in the permanent gamecode banks (bank $00 and $01) to actually implement any code hacks.

[ccovell]

Elmer: programmer by day, typeface fetishist by night. 

[CrackTiger]

Programming isn't the only thing he's a "pro" at.

[esteban]

Elmer: programmer by day, typeface fetishist by night. 

Hahahhhahah.

It is so true, and it is so awesome.

[megatron-uk]

Right, so it looks like all of the assets in banks 0x0A, 0x0C and 0x0E are script. They also coincide with my extracted text regions, so that's great. I can use these new accurate locations to extract the sections in more detail.

Here are the sections so far:

Bank 0x0C

Code Select Expand

Bank: 0xc
---> Contains 12 asset pointers
---> Region 0x18000 - 0x1bfff
---> Starting asset chunk (0x1) located at: 0x18019
---> Finding next asset sequence
-----> 0x1: 0x18019 - 0x18462 [1097 bytes]
-----> 0x3: 0x18462 - 0x18c61 [2047 bytes]
-----> 0x5: 0x18c61 - 0x19147 [1254 bytes]
-----> 0x7: 0x19147 - 0x19927 [2016 bytes]
-----> 0x9: 0x19927 - 0x19c16 [751 bytes]
-----> 0xb: 0x19c16 - 0x1a34b [1845 bytes]
-----> 0xd: 0x1a34b - 0x1ae17 [2764 bytes]
-----> 0xf: 0x1ae17 - 0x1aea2 [139 bytes]
-----> 0x11: 0x1aea2 - 0x1b1d5 [819 bytes]
-----> 0x13: 0x1b1d5 - 0x1b5d8 [1027 bytes]
-----> 0x15: 0x1b5d8 - 0x1b8ce [758 bytes]
-----> 0x17: 0x1b8ce - 0x1bfff [1841 bytes]
Bank 0x0A

Code Select Expand

Bank: 0xa
---> Contains 5 asset pointers
---> Region 0x14000 - 0x17fff
---> Starting asset chunk (0x1) located at: 0x1400b
---> Finding next asset sequence
-----> 0x1: 0x1400b - 0x154fd [5362 bytes]
-----> 0x3: 0x154fd - 0x16a6f [5490 bytes]
-----> 0x9: 0x175c1 - 0x17fff [2622 bytes]
-----> 0x5: 0x16a6f - 0x16c29 [442 bytes]
-----> 0x7: 0x16c29 - 0x175c1 [2456 bytes]
Bank 0x0E

Code Select Expand

Bank: 0xe
---> Contains 8 asset pointers
---> Region 0x1c000 - 0x1ffff
---> Starting asset chunk (0x1) located at: 0x1c011
---> Finding next asset sequence
-----> 0x1: 0x1c011 - 0x1cb4c [2875 bytes]
-----> 0x3: 0x1cb4c - 0x1ceed [929 bytes]
-----> 0x5: 0x1ceed - 0x1d373 [1158 bytes]
-----> 0x7: 0x1d373 - 0x1d5b2 [575 bytes]
-----> 0x9: 0x1d5b2 - 0x1dc86 [1748 bytes]
-----> 0xb: 0x1dc86 - 0x1ebff [3961 bytes]
-----> 0xd: 0x1ebff - 0x1f0ec [1261 bytes]
-----> 0xf: 0x1f0ec - 0x1ffff [3859 bytes]

With bank 0x14 there only appear to be two script assets. Identified by asset index 0x79 and 0x83. Though only 0x83 is referenced by the main asset table in bank 0x01.

0x83: 0x28085 - 0x288e4 [2143 bytes]
0x79: 0x288e4 - 0x2aa15 [8497 bytes]

I've taken the end address of each chunk to be the start address of the next chunk in ram, so these are the maximum sizes of those asset chunks. You can see that 0x79 runs on from 0x83, so one thing I need to find out is how the script in that section is addressed, when asset 0x79 is not directly referenced in the main bank 0x01 offset pointer table, only 0x83 is for bank 0x14.

[megatron-uk]

So to clarify, every single asset chunk in bank 0x0A, 0x0C and 0x0E is directly accessed by the offset pointers in the bank 0x01 table:

Asset bank load table

0e 0e 0e 0e 0e 0e 0c 0e
0e 00 00 00 00 00 00 00
14 0a 0a 0a 0a 0a 0c 0c
0c 0c 0c 0c 0c 0c 0c 0c
0c

Asset index chunk offset table

01 03 05 07 09 0b 17 0d
0f 00 00 00 00 00 00 00
83 01 03 05 07 09 01 03
05 07 09 0b 0d 0f 11 13
15

Each asset bank has a corresponding index offset in the second table. No asset index is without an entry, apart from asset bank 0x14, which only has one index offset: 0x83.

It looks like from index 0x83 (0x28085) until the end of the chunk 0x79 (0x2aa15) is one contiguous script chunk. This looks to be remarkably similar to my existing extractions, which have grabbed text in that region from 0x28086 up to 0x2a930.

[megatron-uk]

Argh, ignore my 0x79 asset chunk comments. It's transcription error - I wrote 0x288E4 instead of 0x2B8E4 as the physical ROM address.

That makes things simpler; just one script location (pointed to by 0x83) in that bank now. Albeit with 10 KBytes of text data....

[megatron-uk]

Since the asset pointed at by 0x83 is the story main script for the game, it should be possible to relocate that one asset to a new bank, to give me another 60% of space to expand it (from ~10KB of Japanese dialogue to 16KB of English to fill the new bank).

That's going to be a massive help.

[elmer]

Elmer: programmer by day, typeface fetishist by night. 

A man's got to have his passions! 

Do I get a superhero name to go with it? 

Since the asset pointed at by 0x83 is the story main script for the game, it should be possible to relocate that one asset to a new bank, to give me another 60% of space to expand it (from ~10KB of Japanese dialogue to 16KB of English to fill the new bank).

That's going to be a massive help.

You've got it beaten now! 

<EDIT>

You don't even need to expand the ROM at all ... I just can't believe how much unused space there is in there!!! 

You've got a completely free 16KB region in the ROM at offset $50000-$53fff (bank $28).

You've also got over 8KB free in the 16KB asset-block at bank $08 where you could rewrite the block to add some new asset chunks into it.

[megatron-uk]

Just made a new commit to github with a Python based extractor tool (extractAssets.py) that outputs one data file per asset chunk. It saves to ./assets/raw by default.

Files are named as ASSETBANK.ASSETCHUNK.dat.

e.g.

0x14.0x83.dat

Now need to write something to load each data file, split the script into strings and then pass back through my translation table that I used earlier. Hopefully have something working for that tomorrow.

There are 26 script asset chunks, allocating a new 16KB bank for each would increase the size of the ROM by 420KB, which, although wasteful, would mean each block of text would have more than enough space to be fully expanded, without worrying whether each of the assets would fit back in to a single bank.

The main story script asset chunk will probably need the most space for expansion, but I had to make savings here and there in a lot of other places, so this would help with those sections, too.

[ccovell]

Elmer: programmer by day, typeface fetishist by night. 

A man's got to have his passions! 

Do I get a superhero name to go with it? 

"The Masked Descender!"

[elmer]

"The Masked Descender!"

Hahaha ... I like it! 

I'll have to think up a Halloween costume to go with it. 

[NecroPhile]

"Wait... that's not a serif.  Dammit, elmer, zip up!"

[ccovell]

"Wait... that's not a serif.  Dammit, elmer, zip up!"

And you would be his sidekick, "Iota".

[esteban]

"Wait... that's not a serif.  Dammit, elmer, zip up!"

And you would be his sidekick, "Iota".

And your nemesis, Signor SanSerif, would go by the nickname "Eunuch".

[elmer]

"Wait... that's not a serif.  Dammit, elmer, zip up!"

You mean that I couldn't walk around and say "Don't you want to touch my dangling serif?" 

But it would still be better to go, in costume, as the intrepid Financial Times reporters (with name badges), Harry Chad, and his wife Penny. After Harry's investigation of the Clinton Foundation, they're now just know to the police as "Hanging Chad" and "Pregnant Chad".

Ba...da...boooom!  ... 

[NightWolve]

Wow, didn't expect all that help by elmer! Looks like megatron is on his way!

[spenoza]

Files are named as ASSETBANK.ASSETCHUNK.dat.

Don't be like me and misread that as ASSBANK.ASSCHUNK

Unless you like to giggle like a child.

Totally understand the reason you'll be out of the game for 18 years or so. I just killed my free time for the same reason this month.