ipl.bin questions

[Psycho Punch]

I'm trying to understand what's going on here. Basically I did a tiny PCE CD example file for myself a while ago but I cheated, instead of trying to understand what goes on with the CD routines I just did a .org $4070 because someone told me it starts there and that's it.

Since I absolutely HATE to have a random magic file (ipl.bin) without knowing how it works, just told to include it, I'm trying to figure out a bunch of stuff here. One of the questions has to do with CDROM'ing in general.

QUESTINOS:
1. What madman chose L-DIS from all games to extract this IPL template for all homebrew games??? ("ｴﾙﾃﾞｨｽ PROGRAM" at 086A lol) Don't curse the whole homebrew community to do NCS level mediocrity forever, bro.

2. Why does the IPL data start at offset 0x0800 in the data track of the game?

3. I know that the string "PC Engine CD-ROM SYSTEM Copyright HUDSON SOFT / NEC Home Electronics,Ltd." is for trying to make copyright law work as their license enforcer (ayy lmao) but what else is checked before executing code?
There's a bunch of text at the beggining of the data track/ipl.bin (credits for the CDROM system?) following by some "junk" bytes. I zeroed them out to test and all I got was a READ ERROR trying to execute it. Is that also checked or is there any relevant program code in it?

4. The IPL 24-bit data pointer (x0800). The official doc simply states "IPLBLK H/M/L - Load start record no. of CD", is that related to the Min-Sec-Frame format I keep hearing about? How does that work? In my compiled mini sample the first byte of code is located exactly 4096 bytes relative to the start of the "track", and my IPLBLK pointer reads 00 00 02. 1 frame = 2048 bytes?

5. The IPL "No. of Records to read" byte. It's set to 16 (0x10), but 16 what exactly? 16 8192 byte blocks? Why 128kb (out of 256 available?), I only specified ONE bank in my .asm file! How does the assembler know which bank goes where on a big project, which ones get loaded, etc?

6. Program LOAD/EXECUTE addresses. Why $4000, why $4070? Is this configurable? Also this implies that it's only a 8kb bank that gets loaded so how do I make sense of (5)?

7. Mostly useless but what's with the 6-byte space after the program title? Bonanza Bros. has "ASK" written on it, so I had to.

tl;dr I imagine that for loading more data from disc past the initial BIOS load I need to understand 4 and maybe 5 so read that pls.

that's it. for now

[Psycho Punch]

Wikipedia: 'a CD-ROM sector contains 2,352 bytes of user data, composed of 98 frames, each consisting of 33-bytes'

I'm lost.

[elmer]

Wikipedia: 'a CD-ROM sector contains 2,352 bytes of user data, composed of 98 frames, each consisting of 33-bytes'

That's just a case of too-much-information ... it's talking about the extremely-clever very-low-level scheme that the CD format uses to encode the data and provide error-correction on a piece of fast-spinning-plastic. Only burning programs (like ImgBurn) need to care about those low-level details.

A CD-AUDIO sector contains 2,352 bytes of audio (as 16-bit sample data).

A CD-ROM sector contains 2,048 bytes of whatever-you-like data, plus a whole bunch of extra error-correction data that only the burning programs need to bother with ... you never see anything more than the 2,048 bytes-per-sector on the PCE, and LBA (logical block address) addresses are calculated in 2,048-bye increments.

Later on, there came the CD-ROM-XA format used by the PS1 and others, but again, you don't need to care about that on the PCE.

I'm trying to understand what's going on here. Basically I did a tiny PCE CD example file for myself a while ago but I cheated, instead of trying to understand what goes on with the CD routines I just did a .org $4070 because someone told me it starts there and that's it.

Since I absolutely HATE to have a random magic file (ipl.bin) without knowing how it works, just told to include it, I'm trying to figure out a bunch of stuff here. One of the questions has to do with CDROM'ing in general.

The ".org $4070" is a HuC decision, and PCEAS automatically sets up the CD IPL for HuC programs.

There are no overrides for ASM developers (there probably should be).

As it is ... when you really start putting together a serious project, you'll soon outgrow pure-pceas and either modify the isolink.c source for your own needs, or write something a bit more sophisticated (if you're not already).


The PCE System Card itself doesn't really care where you load and run your program, you just have to follow the  rules for formatting the IPL, which tells the BIOS where you want to load stuff.

The System Card loads the 2 IPL sectors at $2800-$37FF, and you can load/run your program as low as $3000 if you wish, with no problems.

The LoX games actually load their data at $2800, which works, but if you load lower than $3000, then your load MUST NOT overwrite $DFE0-$DFFF or the IPL code will crash.

You can load as low as $2680, and even $22D0 if you never use the System Card PSG Player, or Squirrel.

If you don't use the System Card PSG Player, then never overwrite ZP $E6 or $E7, because the System Card IRQ routines look at those to decide when to call the PSG code.


QUESTINOS:

2. Why does the IPL data start at offset 0x0800 in the data track of the game?

Because the IPL program code (and credits text) start at offset 0x0000.

0x0000 + 2048 -> 0x0800

The complete IPL is the first 2 sectors in the first .iso track.

3. I know that the string "PC Engine CD-ROM SYSTEM Copyright HUDSON SOFT / NEC Home Electronics,Ltd." is for trying to make copyright law work as their license enforcer (ayy lmao) but what else is checked before executing code?

There's a bunch of text at the beggining of the data track/ipl.bin (credits for the CDROM system?) following by some "junk" bytes. I zeroed them out to test and all I got was a READ ERROR trying to execute it. Is that also checked or is there any relevant program code in it?

IIRC, that string isn't even checked.

But the content of almost-all of the 1st CD sector is checked byte-for-byte to make sure that it matches the one stored in the System Card BIOS.

That 1st sector is loaded at $2800 and contains code that is executed to load your game-program, using the data that you set up in the IPL Data Block in the 2nd sector.

That's the copyright and license hook ... every CD game must include Hudson's copyrighted IPL code or else the System Card won't run it.

Now do you understand why you can just overwrite stuff in there? 

4. The IPL 24-bit data pointer (x0800). The official doc simply states "IPLBLK H/M/L - Load start record no. of CD", is that related to the Min-Sec-Frame format I keep hearing about? How does that work? In my compiled mini sample the first byte of code is located exactly 4096 bytes relative to the start of the "track", and my IPLBLK pointer reads 00 00 02. 1 frame = 2048 bytes?

It's an LBA address ... i.e. the offset from the start of the .iso track in terms of CD-ROM (2048 byte) sectors.

5. The IPL "No. of Records to read" byte. It's set to 16 (0x10), but 16 what exactly? 16 8192 byte blocks? Why 128kb (out of 256 available?), I only specified ONE bank in my .asm file! How does the assembler know which bank goes where on a big project, which ones get loaded, etc?

16 * 2048 byte sectors -> 32KB.

Get used to that 2KB size, and LBA addresses, they're everywhere. 

The IPL will ONLY load into the original 64KB of RAM (banks $80..$87).

You then have to check to see if you are running on the Super System Card and have more RAM, and do your own load of extra data into banks $68..$7F

6. Program LOAD/EXECUTE addresses. Why $4000, why $4070? Is this configurable? Also this implies that it's only a 8kb bank that gets loaded so how do I make sense of (5)?

It's a HuC thing. If you're programming in ASM, you can load/run wherever you like.

Same with the bank layout IPLMPR2..IPLMPR6, in ASM, you can set them to whatever you like.

EXCEPT, please note that that they're stored as values 0..7, corresponding to banks $80..$87.

7. Mostly useless but what's with the 6-byte space after the program title? Bonanza Bros. has "ASK" written on it, so I had to.

Errrr .... because! If you're looking at the docs, you'll see how much space is allocated for the name, and that there are those extra 6 bytes.

It's probably either a "licensed product code" or "licensed developer code".

It's not checked by the System Card.

[Psycho Punch]

Wow, thanks for the detailed answer, elmer, really appreciate it.

LBA encoding is the keyword I was looking for. For what I can gather, MSF format is 99:60:75, but what about LBA encoding? Is it just a regular 24 bit value (0 to FFFFFF), is it a 24-bit BCD index, is it the same as MSF?

I get it, you can point your program code to be loaded anywhere, but what's so special about $DFE0 to DFFF? I thought they were part of the user work area. Only thing I could see this affecting is the PSG Driver but I thought this got mapped back into user space before the IPL program ran your code.

Looks like it's too much of a hassle to load data outside the default PCEAS area but $4000 onwards should give me plenty of space anyway. I just don't get why 32kb is loaded instead of 40 - again what's so special about C000~DFFF?

[elmer]

LBA encoding is the keyword I was looking for. For what I can gather, MSF format is 99:60:75, but what about LBA encoding? Is it just a regular 24 bit value (0 to FFFFFF), is it a 24-bit BCD index, is it the same as MSF?

Yes, it's just a 24-bit binary sector number (where 0 is the start of the .iso track).

I get it, you can point your program code to be loaded anywhere, but what's so special about $DFE0 to DFFF? I thought they were part of the user work area. Only thing I could see this affecting is the PSG Driver but I thought this got mapped back into user space before the IPL program ran your code.

There's nothing special about $DFE0-$DFFF ... it's just where the IPL moves the code that it needs to execute to actually load your program, if you want to load lower than $3000 (and so would overwrite the code in its default location).

They could/should just have moved it into zero-page and executed from there, but they didn't.

As soon as your code is loaded and starts to run, you can do whatever you like with $DFE0-$DFFF.

Looks like it's too much of a hassle to load data outside the default PCEAS area but $4000 onwards should give me plenty of space anyway. I just don't get why 32kb is loaded instead of 40 - again what's so special about C000~DFFF?

Where are you getting the 32KB from?

If it's the PCEAS default, then that's because it is the size of the HuC bootloader code. Then that code actually loads up your HuC program from its list of overlays.

Nothing magical.

[Psycho Punch]

Got it. I'll probably have to figure out a way to join "overlays" together since apparently ISOLINK adds its own data to them... I'm not sure if that's a bug since I can't figure out if overwriting the second byte of my overlay with '1' has any meaning whatsoever and my ADPCM sample has all bytes intact if I'm not mistaken.



(EDIT: It's just a overlay "index", ie if the overlay is the third parameter for ISOLINK it gets a 3 inserted into it (even though the previous layers might have been larger than 2 sectors). This seems to be a HuC thing, I hope that's the only change it does to the compiled overlays, it's annoying but usable. If there are any other stuff that can corrupt my code please let me know.)

Aside from that I think I got the hang of CDROM adressing, got a ADPCM sample loaded through AD_TRANS and AD_PLAY. Seems easy enough to calculate which sector your data is in if you know the sequence which files are thrown at ISOLINK.

[OldMan]

I can't figure out if overwriting the second byte of my overlay with '1' has any meaning whatsoever ...
It's just a overlay "index", ie if the overlay is the third parameter for ISOLINK it gets a 3 inserted into it ...
I hope that's the only change it does to the compiled overlays,...

Nope. ISOLINK also updates the overlay table with the location/size of the overlay(s).

If you are using ISOLINK, it shouldn't be a problem to run an overlay. Just use cd_exec_overlay with the overlay number.

If you are trying to do it by hand, you will need to fill in the overlay table, as well as setting the overlay number for each overlay. Then you  can call cd_exec_overlay.

(see cdrom.asm for cd_exec_overlay function)

[Psycho Punch]

^^^See, that's the problem I have with PCE CDROM stuff. I want to do a project based on the PC Engine CDROM BIOS in ASM to learn how the thing works. I don't doubt that ISOLINK's "overlay table" is useful (especially with HuC projects), but I'm not OK with how hard it is to figure out what goes where. It overwrites my stuff because it's built to handle HuC's output exclusively...

I tried to read cdrom.asm for that cd_execoverlay function and I ended up being more confused. Why "JMP $C000" after loading an overlay? I get that ovlarray is the table, but where is this defined in the dozen inc/asm files in HuC's include folder? Perhaps I'm just an idiot that can't read other people's code but I hate when I have to dig into that kind of stuff because the library has scarce documentation. Say what you want about the Hu7 docs but they have sufficient detail into all aspects of the system, even if they're sometimes confusing to read.

Besides, it feels like a good portion of the code inside the library is just a thin wrapper into the BIOS routines anyway. I feel like the BIOS itself has better documentation so that's what I'm going to use exclusively from now on (or at least until I find out I was an idiot all along because there's a very hard roadblock to solve that's already done inside the default libraries lol)...

I'm ditching the default ISOLINK too, while I don't want any table'ing messing with my code nor default IPL stuff, I do want sector padding for non-overlay files (like ADPCM samples) and 8-second track compliance. Perhaps I'll mod it or create a new program for this, but for now I'll just stick with MSDOS "copy /b" to join overlays and a hex editor to pad .bin files into 2048 byte sectors, which is working OK in emus for now. What about IPL? Well I clipped off half of the IPL.bin file and did this:



Ahh much better. Sorry for the huge rant, I hope no one interprets this as a "I'm too good for what everyone else uses" thing, that wasn't the intention. I just want to do this in a way that my dumb brain doesn't get too confused. For now I simply cannot work with the default libraries because it's too much effort to figure out what goes where.

[OldMan]

Aplogies in advance for the wall of text.

I tried to read cdrom.asm for that cd_execoverlay function and I ended up being more confused.

Welcome to my world, 2 years ago. There are still parts of the startup I don't understand...

Why "JMP $C000" after loading an overlay?

See startup.asm around line 263.

Overlays have a different execution address, presumably so they can check for problems.
Fwiw, I -think- that's also why the ipl execute address is $4070  (startup.asm, line 230)

I get that ovlarray is the table, but where is this defined in the dozen inc/asm files
in HuC's include folder?

Again, it's in startup.asm, line 168. First thing in DATA_BANK (which is bank 3 for cds, I think).
It's also not something I want to fill out by hand.

Besides, it feels like a good portion of the code inside the library is just
a thin wrapper into the BIOS routines anyway.

A lot of it is, when building cd's. Many of the bios routines are duplicated
specifically so they can be used for HuCards.
[Though I'd like to shoot whoever thought using ifdefs was a good way to do that...
 I'd prefer startup.asm just have the ifdefs and include cd-specific/HuCard-specific versions.
 But I'm like that.]
 

I'm ditching the default ISOLINK too, while I don't want any table'ing messing
with my code nor default IPL stuff, I do want sector padding for non-overlay files
(like ADPCM samples) and 8-second track compliance. Perhaps I'll mod it or create
a new program for this, but for now I'll just stick with MSDOS "copy /b" to
join overlays and a hex editor to pad .bin files into 2048 byte sectors,
which is working OK in emus for now. What about IPL? Well I clipped off half of
the IPL.bin file and did this:

What I ended up doing was splitting the HuC ipl.bin in two, like you did, and renaming the
credits/boot code to ipl1.bin
Then I created an ipl2.asm which held the ipl table (and some extra code),
assembled it, and copy /b ipl1.bin+ipl2.bin ipl.bin. That put the ipl.bin file back.
(I may have chopped ipl2.bin up to get just the 2K I wanted. Don't remember)

I'd have to check bios for the exact address, but part of the first ipl sector
(which is checked for changes, unfortunately) is the 'real' boot code. It executes from
RAM (which makes more sense to me). It can do some neat things, like auto-loading adpcm
data and/or graphics and/or a splash screen. It's worth looking at a disassembly to see
how it operates.

Ahh much better. Sorry for the huge rant, I hope no one interprets this as a
"I'm too good for what everyone else uses" thing, that wasn't the intention.
I just want to do this in a way that my dumb brain doesn't get too confused.
For now I simply cannot work with the default libraries because it's too much
effort to figure out what goes where

Understood. And seconded. It's a pain, even in HuC, to get things 'just right'.