My progress on PC-FX homebrew development

[gameblabla]

I was always interested in the PC-FX but the state of liberis unfortunately prevented me from doing anything
besides a very basic game, Simple Battle FX, for it.

The official SDK that was released for the PCFX-GA was certainly interested
but was said to only work on said hardware without modifications.
Besides, it only works on DOS/V (it always returns DOS16M errors for me) so this was a big pain in the arse.

I'm happy to say however, that i'm starting to get a bit more familiar with the (quirky) hardware.
Well yes and no lol... i'm not exactly experienced in all of this.

I wish i had discovered this older thread as Oldrover's posts and code samples were very interesting and
it's what helped me get CD-DA and ADPCM working in the first place.
https://www.pcengine-fx.com/forums/index.php?topic=18435.50

This is what was possible with liberis so far :
- Text on all background layers
- Simple 16 colors background image loaded from CD directly to KRAM
- Controllers (although it is apparently not working properly without a hack)
- Sprites (i'm having a bit of trouble with the 7up examples however)
- PSG soundchip (very low level functions)

However as you can see, this was not enough for me to make, say, a basic visual novel game.
Besides, the NEC PC-FX really needs Adol Christin being stuffed things up his ass.
In order to achieve this noble goal, i set out to find out more about the hardware.

32bpp

Initially i wanted to just go ahead with the 16 colors example that was there for a while.
And it has been confirmed to work on real hardware as well.
Unfortunately i had no idea how he converted his picture from a BMP or similar. When i asked him on twitter,
he said that he couldn't remember other than the fact it was his own tool.
Things were not going so well so far.

I took a look at GMAKER's tools and i eventually found b2k.
This tool however only supports the 16 & 32bpp color modes, not the 256/16 color modes. (this will be a problem as you will find out later)
The tool thankfully does work under dosbox and i was able to get something out of it.

However, there was still the problem of actually getting the whole thing to work in the first place.
I went under the assumption that the 16 color example just needed some modifications to it for it to really work.
I was able to get somewhere but it would only parse part of my image on the screen.
I immediatly suspected the microcode but i was not familiar with it so i read the translated documentation here :
http://daifukkat.su/pcfx/

Eventually in the KING section under Common microcode, i found these :

Code Select Expand

16M color mode / 64k color mode microprogram
Code Operation
0000 +0 from CG is accessed
0001 +1 from CG is accessed
0002 +2 from CG is accessed
0003 +3 from CG is accessed
0004 +4 from CG is accessed
0005 +5 from CG is accessed
0006 +6 from CG is accessed
0007 +7 from CG is accessed

16 color mode microprogram
Code Operation
0000 +0 from CG is accessed
0001 +1 from CG is accessed

The 16 color mode microprogram was suspiciously similar to the one that was in the example.
What if i reimplement it as they said instead ?

So yeah, this is basically how i got my 32bpp image to work.
I should note that there is also support for 16bpp but as you can see, it seems to take the same amount of memory for some bizarre reasons.
So there does not seem to be much point in using it.

I later figured out 8bpp and also wrote a tool for converting BMP palettes to a C array suitable for my needs.

CD-DA/ADPCM

In the same thread, Oldrover posted a bunch of code on how to playback an ADPCM sample as well as CD-DA.
It was mostly straight forward to get his code up and running though as he said, there were a few issues which i have now fixed.

As for AD-PCM, it was working fine but i can't get my head around the bits so for now it's only working at 16khz and the first channel. (the PC-FX has 2 channels and can support up to 32Khz)
Using the wav2vox tool, it was sounding OK but not particularly great... In fact, this sounds much worse than ADPCM on the Dreamcast.
The lower frequency could be the reason why but oh well.

Conclusion

All of my code is here if you want to take a look or even use it :
https://github.com/gameblabla/liberis/tree/cdrom

We are not exactly covering the whole hardware yet.
For example, the MJPEG side of things is still a mystery and woefully documented.
I don't think i can get MJPEG working on that thing sadly, even if it would be nice.

Doing software rendering (ala VGA) for 3D on the PC-FX would be nice but this would require converting the colors from RGB to Y8U4V4..
As a result, a performance penalty may occur.
Perhaps it wouldn't be so bad in 16/256 colors mode but the routine elmer posted would be too slow on the hardware.
I'm not even sure why i'm even talking about this when we don't even have a proper open source image converter for it lol...

There are some examples for the 7up/VCE/VDC chips but i'm not too familiar with those.
Most retail games use them because honestly, they don't have much of a choice given the constraints the KING chip has.

I'm also looking for artists etc... potentially interested in my project. : P
As it is, i don't think i can really port a game like Evil australians unless i mess around with the 7up chip. (the KING chip doesn't really have sprites and doing software rendering for small sprites is well, probably not a good idea lol)

Sidenote : Consolizing the PCFX-GA would be nice but it seems that it could only be easily done with the DOS/V version, which is unfortunately not cheap and rare.
Perhaps instead we could desolder the 3D chip from it and manually wiring it up to a PC-FX. This sounds like a terrible idea though, i have to admit.

I will get my PC-FX (as well as my 3DO) in a few months so until then, i won't be able to test on real hardware. (I need to try in particular the input code as it's said to be partially broken in liberis)

[gameblabla]

I figured out how to play a CD-DA track and loop it.
It turns out the PC-FX was not fully compliant with the SCSI-2 standard !
This is apparently because it was based upon an earlier (free) draft of the standard.
Instead, if you want to have a CD-DA track looping, you have to use the vendor-specific 0xD8 and 0xD9 modes.

The first one specifies the starting track and the other one specifies if it should loop or not and the ending track.
So yes, if you feel like it, you can make it play through 2 tracks and then loop.
I was able to figure this one out thanks to Mednafen's source code on the matter. (and i used mednafen to catch the SCSI calls games were making to figure it out)

The next issue i was encountering when using those is that there is a bug within liberis itself with the low levels SCSI commands : they would just freeze the software or be ignored if a second call to it was made.
I have been told to look into the cd_read and the other high level functions that Marshall worked on as those work fine.
There was the issue of me not knowing any V810 assembly but then i realized that i could simply make a copy of the known working functions, have GCC compile my C code and output it as V810 assembly and then transplant that onto my new function.
Shitty idea actually worked it seems and it's no longer freezing the hardware.

Why it's doing this, i'm not too sure :
I had taken a look at it and it seems to check for the CD-ROM drive status before it goes on.

In any case, the only thing i have to do is make the second ADPCM channel work properly.

[NightWolve]

I figured out how to play a CD-DA track and loop it.
It turns out the PC-FX was not fully compliant with the SCSI-2 standard !
This is apparently because it was based upon an earlier (free) draft of the standard.
Instead, if you want to have a CD-DA track looping, you have to use the vendor-specific 0xD8 and 0xD9 modes.

That's interesting, I had some experience/difficulties in this area when I was developing TurboRip!

The 0xD8 read CD-DA/raw command is a very old fallback on some early NEC (and Toshiba?) PC CDROM drives that were made before SCSI-3 AKA "MMC [1] (Multi-Media Commands)" was drafted/adopted which also dealt with DVD commands!

Ultimately they decided on the 0xBE command to read data sectors raw or for CD-DA extraction/reads, etc. Unfortunately it created an incompatibility with drives produced with the 0xD8 command unless the CD ripping software accounted for it. So all the manufacturers that took early initiative to produce such CDROM drives with 0xD8 kinda got egg on their face and had to rework the designs.

Exact Audio Copy is pretty good about this; it has a specific Drive Option to detect the correct command (just in case), but yeah, pretty much MMC1/0xBE became universal and it's those really rare cases it might be a problem. That's what David Michel had told me years ago when I first started working on TurboRip, to just deal with understanding 0xBE.



Starting with Windows NT, you still had the option to build your own SCSI command packet and send it to the drive direct, but also the prescribed NT SPTI method. In Windows 7 they forced you to do this for MODE1 data sectors which broke TurboRip and I'm assuming other CD ripping software because Microsoft removed this "security" requirement in Windows 8/10+. So I tried to use this exclusively, only to discover from Chris Covell's feedback/tests that it was extremely slow! Part of the reason is if you looked at the API, you can find it checks for old [NEC] drives that need the 0xD8 command, so I liked the idea of more guaranteed compatibility which is why I tried to make it default! In the end, I did a balance: defaulted to my own SCSI command packets with 0xBE, then the SPTI method upon failure detection (which IS hard in SCSI!!). I kept the speed, and still gained some extra compatibility (possibly)!

Anyway, I'm afraid I can't help add much else to your research it being the PC-FX and all, but yeah, I do have some experience with general SCSI-3/MMC commands/programming under a Win32 environment so I share when I can.

[gameblabla]

Well i'm glad that i don't have to go through that SCSI trouble...

There have been a few updates since then :

- I was able to come up with a tool for allowing liberis executable to boot from backup memory.
The only limitation is the size being limited to 2MB because you cannot execute code from ROM
and it has to be copied to RAM first.

My tool is released here as source code (i can compile a windows executable if anyone is interested) :
https://github.com/gameblabla/bin2backup

- Thanks to another person, i was able to get my hands on GMAKER PLUS and its AGE executable.
This was needed as i have no other means rn to convert graphics to 8bpp.
This means that i can now pretty much cover all graphical modes (except for the "block" mode ones that i have yet to try)

This is all here : https://github.com/gameblabla/liberis/tree/cdrom

- I also rewrote oldrover's pal2yuv tool as his tool was wonky to use and annoying for liberis development.
I rewrote one so that it can take a palette from BMP files and export it as a C array for use in liberis.
This can be found here : https://github.com/gameblabla/bmp2yuv



There is still a lot to be figured out but so far, i've made some nice progress.
I hope this make other people interested as well.

[gameblabla]

Another PC-FX dev update :
Figured out the 16bpp KING BG format as well and came up with another tool that improves the image quality over the official SDK tools.
Unlike 8bpp, bytes are not swapped but it still expects Y8U4V4 pixels.
Tool is here : https://t.co/XBosYJv0Kr pic.twitter.com/7AvTkf7i3m

— gameblabla (@gameblabla) January 28, 2022

Some newsworthy progress !
Finally decided to take a closer look at the 8bpp and 16bpp KING BG formats to see what the hardware expects.

8bpp
For 8bpp, you need two things :
- A 256 colors palette with Y8U4V4 colors.
- An 8-bits buffer that is byte-swapped (big endian)

Even though the CPU (V810) is little endian, KING's 8bpp mode expects a big endian framebuffer.
This makes it really annoying to work with, as not only you need to convert from RGB to YUV,
but you also need to swap the bytes manually.
If you're not careful about it, this can result into a noticeable performance penalty.
Given that the CPU isn't that strong compared to the Saturn's SH-2, you ought to be careful

16bpp
For 16bpp, no color palette is needed (obviously).
However, it still expects Y8U4V4 pixels.
Unlike 8bpp, framebuffer is little endian so it does not need to be byte swapped. (Good)
However this may or may not be ideal for software rendering as it consumes twice as much memory.

Based on my findings, i've released a (better) tool for 8/16bpp so i don't have to use the official SDK tools just to convert an image.

github.com/gameblabla/bmp2huc

For 16bpp, image quality is greatly improved compared to b2k.
Check this out :

b2k	BMP2HUC (my own)

I'm not sure how they screwed up that badly but i must assume they must do some internal converting down to RGB555 or something before converting to Y8U4V4.
Due to the YUV nature of the PC-FX, it's a better idea to convert from 24/32bpp rather than 16bpp directly.
Of course though, if you are doing software rendering on the hardware itself, then you should go directly with 16bpp.