[This is a not very good translation from Japanese, I am deeply sorry] --------------------------------------------------------------------- About the PIC format 1993-09-26 Yanagisawa * Introduction Because I had not written a decent specification for the PIC format, I'm afraid I caused a lot of trouble for everyone porting the algorithm. There were actually quite a few poor specifications, so you could figure it out with some effort. It was troublesome to keep saying there's no documentation, but there just wasn't a decent specification to publish. (^^; The specification is summarised to some extent, so please forgive me (^^; especially since I may not be remembering some things correctly. (oooi) If you notice any strange things, please let me know. It might be difficult to understand because some parts are vague, please forgive that as well. I also enclosed the source of a loader written in C, but it is almost the same shoddy version that was published in the Oh!X magazine. In X68K 15-bit color the image size is only 512x512. Please consider it only a supplement to help understand the PIC algorithm. * Pronunciation It is formally "Pikku", but it's not a problem to say it differently. (^^; * Format - overview and header From the beginning of the file, as follows: 'P' 1 byte PIC signature 'I' 1 byte 'C' 1 byte any bytes Optional Comment section except $1A $1A (EOF) 1 byte End of comment section any bytes Optional Dummy data except 0 0 1 byte True end of comment section 0 1 byte Reserved x 1 byte bit 0..3: Model type bit 4..7: Mode for each model x 1 word Color bitdepth x 1 word Image width in pixels x 1 word Image height in pixels p nbyte Palette data etc. The content depends on model/mode/color depth (Original, nothing here) The compressed data follows. In addition, words are 2 bytes long, and values are big-endian (Motorola). * Description of compression Here I will explain the X68k 15-bit color format (basic PIC). Other formats are basically the same, so I will just point out the differences later. Firstly, compression algorithms can be completely lossless, or lossy (omitting image information without visual problems). PIC is a lossless algorithm. And, since it is very difficult to compress every image with the same algorithm, it is important to clarify what kind of images are being targeted. For PIC, best results are probably gained for anime-style images. (Other images will work too, but the compression level will not be as good as other algorithms.) With a target image decided, it is compressed using its characteristics. In the case of anime-style images, pixels of the same color appear in continuous runs both horizontally and vertically. This characteristic allows for compression. Run-length encoding is a simple example. In this, you follow pixels from left to right, recording any continuous single-color runs, but the compression ratio isn't very high. However, the idea of recording points where the color changes is also the basis for PIC. In other words, the challenge is how to most efficiently record positions where the color changes, and the color data itself. First, let's consider how to record change points. While run-length encoding only looks at horizontal runs, it is natural to also consider vertical runs. So we are effectively following contours in images. For example, in a picture like Figure 1, the change points are extracted first. +++#### +--#--- +--#--- ++++### +---#-- ------- +++++## +----#- ------- +++++## +----#- ------- ++++### +---#-- ------- +++#### +--#--- ------- Figure 1 Figure 2 Figure 3 With just the change points, it will look like Figure 2. As you can see, the change point of # meanders toward the bottom. In PIC, this is called a chain. Chains are recorded separately to reduce the total number of change points. In Figure 2, the chain for # will be recorded from the second line downward as "right-right-down-left-left". The + will be "down-down-down-down-down". With the chains removed, only two change points are left in Figure 3. As you are following a chain, look at the two pixels on the below left and below right of the current position, and if the change point uses the same color, add it to the chain. Repeat along the chain as far as possible. This chain is encoded as shown in Figure 4. Note that 0/1 is encoded as a single bit, but 01 is encoded as a two-bit value. no chain ---+------> 0 --------------------------------------> end chain | ^ | yes | -------> 1 -------+---> 00 -------+---> 0 ---| ^ | | | | | | | ----> 1 ---+---> 0 --->| | +---> 01 --->| | left 2 | | | left | | | | | | | | | | | ----> 1 --->| | +---> 10 --->| right 2 | | | straight | | | | | | | | | | | ----> 11 --->| | | right | | | | | |---------<------------------<-------------- Figure 4 Next the change point position is recorded, as the horizontal distance from the previous change point. The length is encoded as shown in Figure 5. 1-2 0x 3-6 10xx 7-14 110xxx 15-30 1110xxxx 31-62 11110xxxxx 63-126 111110xxxxxx 127-254 1111110xxxxxxx 255-510 11111110xxxxxxxx ... ... ... ... ... ... Each x is a single 1 or 0 bit. Figure 5 For example, to record the distance 2, it would be two bits: "01". Shorter distances are encoded with fewer bits, so it is reasonably efficient. Finally, as for recording the color, simple caching can be used by noting that the same color tends to occur around the same neighborhood. Specifically, we'll keep the past 128 color differences in a table. If the next color is already in the table, record a flag indicating it is in the table and the position in the table. If it is not in the table, record the flag and the color code. If the same colors appear frequently, this should minimise the bits required. The encoding is shown in Figure 6. color ---+---> 0 ---> 15-bit color code ----------------> | | ----> 1 ---> 7-bit color table position ---| Figure 6 In addition, when a color is not in the table, the oldest color in the table is discarded and replaced with this new color. It is the Least Recently Used method. The specific method is here. Please refer to it as I will update the C source separately. (^^; The above is summarised in Figure 7. begin -------> length ---> color ---> chain ---+---> end ^ | | | |-------------------------------------- Figure 7 In other words: 1. Find all change points in the image. (The end of each row connects to the beginning of the next line.) 2. Find a change point, and record the length between change points. 3. Record the color of the change point. 4. Check if a chain can go downward from the change point. -- If a chain is possible -- 4a. Record the position of the chain, and remove the change point. 4b. Trace the chain downward. 4c. Repeat until the chain can't continue further. (The chain does not cross the left and right edges of the image.) 5. Repeat from 2. Like so. Once there are no more change points in step 2, the image has been compressed. Decompression is the same in reverse. (^^; * PIC differences The PIC format is slightly different depending on the characteristics of each computer model. Why make different loaders? Perhaps for some models, a specialised format is too convenient and difficult to throw away. (Well, it might have been different if there were as many machines as we have now.) First there is a model type in the header. x 1 byte bit 0..3: Model type bit 4..7: Mode for each model The lower 4 bits indicate the model, as follows: Model type | Model name ================+=============== 0 | X68k 1 | PC-88VA 2 | FM-Towns 3 | Mac $F | Generic ----------------+--------------- The basic compression method is always the same, while only the ratios, color depth, and presence or absence of a palette are different. The differences are explained below. Where palette data is present, it is saved between the image size and the compressed data. When the palette is described as GGGRRRBBBI, that means 3 bits of Green, 3 bits of Red, 3 bits of Blue, and 1 bit of Intensity from the topmost bit. The Intensity bit will act as a shared lowest 4th bit for all RGB components. So, in the said description, the red component is RRRI, green is GGGI, and blue is BBBI. Since the palette interpretation depends on the monitor being used, please consider it an approximate color. * Model type 0: X68k The original PIC. Since the extended APIC format only uses more colors, both can be explained together. The size is variable. The mode information is always 0. Number of | Screen | | Color | colors | ratio | Palette | caching | Misc. ===========+==========+==================+===========+========= 16 | 1:1 | GGGGGRRRRRBBBBBI | No | *1 256 | 16:9 | GGGGGRRRRRBBBBBI | No | 32768 | 16:9 | Not present *2 | Yes | 65536 | 16:9 | Not present *3 | Yes | -----------+----------+------------------+-----------+--------- *1 If the image is taller than 512 rows, the chain buffer only needs to fit 512 rows. When compressing, chains aren't allowed to cross the 512th row, so as to save memory by treating tall images as two halves. This doesn't change anything on the decompressing side. *2 The color always uses 15-bit values, GGGGGRRRRRBBBBB. (I=0) *3 The color always uses 16-bit values, GGGGGRRRRRBBBBBI. Omake part 1: When pic.r says "can't read picture error" trying to load an APIC image, that's because pic.r expects a few extra bytes of data, even if the compressed data actually fits in fewer bytes, due to how it is buffered internally. However, APIC has different internal buffering, so it can use an exact data size and doesn't output the extra bytes, causing pic.r to generate an error. Omake part 2: Square PICs. This uses a 512x512 resolution, with 1:1 ratio. Further information will be embedded in the header's comment section. Since the comment is displayed in the image library of NIFTY/Fgalav, it is good to consult that as necessary. * Model type 1: PC-88VA Various methods are adopted for 88VA. First, the common features. About the ratio: On the 88VA, PIC doesn't have variable image sizes. It is always a fullscreen image. Therefore, please consider the image size as the native screen size. The basic ratio is the same as PC-98, about 1:1 in 640x400 mode. For example, an image of 640x200 should be understood as relative to 640x400, so pixels are twice as tall. However, there is an exception. In HR mode, the full screen size is half of the monitor. The image displayed will be only 320x400 in 640x400. About image size: The 88VA has many screen modes. Please support all sizes if possible. 640x400 (256 colors) 640x204 640x200 320x408 320x400 320x400 (HR) 320x200 Additionally, sizes like 200x640 (normal and reverse) are possible with the 88VA, but since it is unlikely that UP will be done as image data, don't worry if you can't support such modes because it's too troublesome. HR mode: When the lowest bit of mode is set (0001), HR mode is engaged. In this mode, 320x400 is 1:1. Summary: Number of | Screen | | Color | colors | ratio | Palette | caching | Misc. ===========+=================+==================+===========+========= 256 | Depends on size | Not present *1 | No | *4 4096 | Depends on size | Not present *2 | Yes | 65536 | Depends on size | Not present *3 | Yes | -----------+-----------------+------------------+-----------+--------- *1 The color always uses 8-bit values GGGRRRBB. *2 The color always uses 12-bit values GGGGRRRRBBBB. *3 The color always uses 16-bit values GGGGGGRRRRRBBBBB. *4 There are two 256-color modes. One has a normal 256 colors. The other has seemingly 64k colors, but is really a 256-color mode. Although the header marks it as 64k colors, use 256 colors when the second-lowest bit of the mode is set (0010). This allows to efficiently compress dithered data. Generally, dithering looks like the below pattern. When compressed with PIC, the efficiency is not so good. .x.x.x.x.x 0 x.x.x.x.x. 1 .x.x.x.x.x 2 x.x.x.x.x. 3 .x.x.x.x.x 4 x.x.x.x.x. 5 This needs to be transformed. Move the odd rows to the right of the even rows, as shown. 0 .x.x.x.x.xx.x.x.x.x. 1 2 .x.x.x.x.xx.x.x.x.x. 3 4 .x.x.x.x.xx.x.x.x.x. 5 Save pixel pairs as a single pixel. (Change 256 colors into 64K colors.) [.x][.x][.x][.x][.x][x.][x.][x.][x.][x.] [.x][.x][.x][.x][.x][x.][x.][x.][x.][x.] [.x][.x][.x][.x][.x][x.][x.][x.][x.][x.] To make it clearer, let [.x] be ★, and [x.] is ☆. Then: ★★★★★☆☆☆☆☆ ★★★★★☆☆☆☆☆ ★★★★★☆☆☆☆☆ Looks much clearer this way, as you can see. As a result, the original 8-bit-color 10x6 image became a 16-bit-color 10x3 image. So the color is twice as big, the image width is unchanged, and the image height was halved. Record this. Please note that the header will say it uses 16-bit colors, and the vertical size is halved. Although the image is decompressed as if it used 16 bits, the buffer should be treated as containing 8-bit colors when actually displaying the image. Incidentally, the 16-bit color will have reversed upper and lower bytes, so 680xx series users will need to be careful. Omake: Although it's two screenfuls of data, since two screens in HR mode takes quite a lot of memory, users will be pleased if they can display the image on just one screen. * Model type 2: FM-Towns It is the same as X68K except that the ratio is 1:1. The mode will indicate whether to use screen mode 5. This is usually $0C/$05, where $0C means a normal-resolution mode, and $05 is a low-resolution mode (320x240). Image loader authors don't need to be particularly aware of the mode, but if you double the image width and height, it may feel more correct. Additionally, there are only 32768 colors, and the arrangement is the same as X68K: GGGGGRRRRRBBBBB. * Model type 3: Mac The screen size varies, and the ratio is 1:1. For now, the color depth is always 15 bits, in the order RRRRRGGGGGBBBBB. * Model type $F Model type $0F is for most recently extended PIC. It is mainly used for 1:1 ratio with 256-color data. It can also be used with 16 million colors. Also note that the header is slightly different compared to other models. Mode: The ratio is saved instead of the mode. Mode | Ratio ==========+========= $00 | 1:1 $01 | X68k at 512x512, ratio 16:9 $0F | Use later ratio information ----------+--------- Extended header: After the image height, additional data follows: (word = 16 bits) - 1 word: Image coordinate X (not specified by -1) - 1 word: Image coordinate Y (not specified by -1) - 1 word: True image ratio (top byte = horizontal, lower byte = vertical) ($0101 or $0D09) - 1 byte: Palette bit length (only present for 16/256 colors) - Pack the palette in GRB order, according to bit length - Compressed data comes after this. Note that even if the palette ends with space left over for a few bits, the compressed data only begins at the next byte. No other byte alignment is done. * Although it is related to the mode ratio, normally you can ignore this and just use the ratio from the mode. But if you want to use an exactly correct ratio, use this. Summary: Number of | Screen | | Color | colors | ratio | Palette | caching | Misc. ===========+=================+==================+===========+========= 16 | By the header | GRB-order *1 | No | 256 | By the header | GRB-order *1 | No | 4096 | By the header | Not present *2 | Yes | 32768 | By the header | Not present *3 | Yes | 65536 | By the header | Not present *4 | Yes | 24-bit | By the header | Not present *5 | Yes | 32-bit | By the header | Not present *6 | Yes | -----------+-----------------+------------------+-----------+--------- *1 Number of bits depends on the header *2 The color always uses 12-bit values GGGGRRRRBBBB *3 The color always uses 15-bit values, GGGGGRRRRRBBBBB *4 The color always uses 16-bit values, GGGGGRRRRRBBBBBI *5 The color always uses 24-bit values, GGGGGGGGRRRRRRRRBBBBBBBB *6 I have not decided yet Caution: There is quite a lot of stuff in this format, but there are also various elements from PIC2, and I don't think that everything is actually used. So if you are planning to make a compressor for anything other than 1:1 ratio 256 colors, please make sure to contact us on Nifty/Fgalav, so that it will be easier to create decompressors for it. * Conclusion Thanks to everyone who ported PIC to different models. And thanks to everyone who is using PIC. I'm afraid I may be a bothersome sloppy author, so please forgive me. Thank you for continuing to support PIC, yoroshiku onegaishimasu. Reprinting and quoting this material is freely permitted (no contact is necessary). In addition, you are free to create compressors and image loaders. Nifty NCD01745 PC-VAN LVB06263 Yanagisawa