Pi and PIC graphics formats

JapaneseEnglish
Pi techPi tech
PIC formatPIC format

Various Japanese specifications and native X68000 software can be found here. The table on the right links to the specifications.

The open-source Recoil project specialises in handling many old image formats such as these. They have a handy browser-based viewer here.

Grapholic and Susie may be able to natively view Pi and PIC files. The ViX tool for Win9x/XP can also view and convert Pi and PIC files.

My own code for opening these files is on Github as part of SuperSakura. You can convert Pi and PIC files to PNG with the "decomp" tool. There is a win32 binary of the whole project here, which contains decomp.


Background

The Pi and PIC formats were developed around 1990 by the clever programmer Yanagisawa-san, of whom I can't find any information apart from the name. PIC was targeted at the high-color-capable X68000 computers, while Pi was optimal for the reduced palette used by the PC98.

Although Pi uses a strictly superior algorithm compared to MAG, the latter's slight headstart had already established it as the standard file format for artists and fans sharing images on BBSes. It also can't have hurt that the developer of the MAG format also created the most popular PC98 image editor. And in a few years, GIF with its on average even better LZW compression took the crown, while JPGs eclipsed PIC and PIC2. (Also, don't confuse these with Apple's PICT format, which is a whole other thing.)

The Pi format did see use by Japanese software houses, who needed the most efficient available image compression to fit their games on fewer diskettes. I reverse-engineered the format without any documentation back around 2006, having grown curious about classic visual novel engines. Curiously, although clearly many others have written Pi decoders, there's still a dearth of English specifications, so I'm putting mine out, with additional detail gained from the original Japanese specifications.


Pi Specification

Pi header

(MSB-first)

OffsetSizeDescription
02Signature "Pi" (sometimes omitted or zeroed out)
2.. Comment and metadata section, encoded in Shift-JIS; Variable-length, terminates with byte $1A, and the first 00 after $1A marks the start of the real header
01 Start of header, always 00
11Mode byte, usually 00
21Pixel aspect ratio X
31Pixel aspect ratio Y
41Bitdepth
54Compressor model signature
92Size of compressor-specific data, usually 0
11..Compressor-specific data section
..2Image pixel width
..2Image pixel height
....Palette: 16 or 256 byte triplets, order RGB

The mode byte's top bit is supposed to indicate that a default palette be used, but even my test images with a $FF mode still include the palette. You can maybe ignore this.

The pixel aspect ratio X:Y should signal whether any image stretching will be necessary after decompressing, but none of my test images make use of this feature. The ratios 0:0 and 1:1 both indicate a normal 1:1 pixel aspect ratio.

Bitdepth is 4 for 16-color images, and 8 for 256-color images. You'll need this to know how much palette data follows the header. If bitdepth is $FF, try defaulting to 16 colors.

The compressor model can be ignored, except to take into account how many colors the source system was capable of displaying. The X68K has 5 significant bits in its palette components, while the default PC98 systems have 4.

Compressor-specific data is hardly ever used, and I don't know how to interpret it. Doesn't seem to affect the end result, so skip it.

Every palette component byte should have its most significant bits copied to the rest of the byte, much like the MAG format does. The visual difference is minor even if you use the palette bytes directly as 8-bit values.

Pi decompression

Click the image to download the example Pi file, to test your decompressor.

To start with, create an output buffer and a delta table. The algorithm will be using a combination of delta codes and sequence repetition to fill the output buffer.

The output buffer should be 1 byte per pixel, and the exact size specified in the header. The image's bitdepth makes no difference here.

The delta table is like an array of linked lists, showing for each color byte a list of most recently associated color bytes. At a palette size of 16 colors, the table must be 16x16 bytes; at 256 colors, it must be 256x256 bytes. The table must be initialised using this:

table[x, y] = (number of colors + x - y) modulo (number of colors)

An 8-color table for example would look like this (note the reversed axes):

x \ y76543210
012345670
123456701
234567012
345670123
456701234
567012345
670123456
701234567

So, whenever you need to process a delta code:

Example: From an initial state using the above 8x8 table:

Input delta code sequence
574110011001
Output color bytes
340034003400
Delta table afterward
x \ y76543210
012456730
1(no change)
2(no change)
356701234
456712340
5(no change)
6(no change)
7(no change)

As you can see, any repeating pixel patterns in the image will quickly reconfigure the delta table so the pattern's colors can be encoded using low-numbered delta codes, mostly 0's and 1's.

In the compressed stream, delta codes are stored using variable bit lengths, with the smallest codes assigned the shortest lengths. You'll need to read the delta codes one bit at a time, until you have a valid code. There are two different encodings, one for 16-color images, the other for 256-color images. Use the right one indicated by the header's bit depth.

4-bit delta encoding
Binary encodingDecimal delta code range
1x0-1
00x2-3
010xx4-7
011xxx8-15
8-bit delta encoding
Binary encodingDecimal delta code range
1x0-1
00x2-3
010xx4-7
0110xxx8-15
01110xxxx16-31
011110xxxxx32-63
0111110xxxxxx64-127
0111111xxxxxxx128-255

In the compressed stream, delta codes alternate with sequence repetition commands. Each repetition command has a relative location to repeat output from, followed immediately by a repetition length. The length is specified in byte pairs, so for example length 1 = 2 bytes, and 8 = 16 bytes.

There are five possible location codes, each of which is two or three bits long, as follows:

Repetition lengths are encoded with bit sequences like this:

Repetition length encoding
Binary encodingDecimal output range
01
10x2-3
110xx4-7
1110xxx8-15
11110xxxx16-31
......

All repetition commands are assumed to be immediately followed by another repetition command, unless the next repetition location code is exactly the same as the immediate prior location code.

Example repetition sequence:

Input repetition commands
LocationLengthLocationLengthLocation
110110100110001
Meaning
Copy 6x2 bytes from one row above and one byte ahead Copy 2x2 bytes from one row above No more repeats

Be sure to guard against repetition lengths going beyond the end of your output buffer. I have some test images that will crash an unguarded decoder.

To decompress an image bitstream, follow these steps:

Example bitstream from the beginning of the below 16-color image:

Input stream (hexadecimal)
5D 29 80 26 75
Input stream (binary)
0101-1101 0010-1001 1000-0000 0010-0110 0111-0101
Input codesAction
01011Delta code 7: output color 9 (jade green)
10Delta code 0: output color 9
10Repeat from two rows above...
0... 1x2 bytes, except this is the first repeat, so it's 0x2 bytes (nothing)
10No more repeats
10Delta code 0: output color 9
011000Delta code 8: output color 1 (burnt umber)
0Going to repeat...
00... the last 4 output bytes...
0... for 1x2 bytes: output color 9 and 9
00No more repeats
10Delta code 0: output color 1
011001Delta code 9: output color 8 (dark teal)
1More delta codes follow...
10Delta code 0: output color 8
10Delta code 0: output color 8
1More delta codes follow...



PIC Specification

tbd