Japanese specification for Maki v1: http://www.jisyo.com/viewer/faq/maki_tech.htm
Japanese specification for MAG v2: http://www.jisyo.com/viewer/faq/mag_tech.htm
Japanese history of the MAG format at a doujin dictionary: http://www.paradisearmy.com/doujin/pasok4u.htm
Alternately, see here:
... which links to a browser-based MAG viewer here. Just drag and drop a picture file in the browser window! It'll only open MAG v2 files, not MAKI v1 files.
Finally, the open-source Recoil project specialises in handling many old image formats such as these.
Back in the day before LZW compression became ubiquitous, less efficient image compression methods were used. Woody RINN's MAKI format (or, more familiarly, Maki-chan graphics) became popular for images in Japan, and was used for many classic games. Rinn-san is actually a capable illustrator as well as programmer, and has a few commercially published mangas. He developed the Maguro Paint System (MPS), which became the most popular PC-98 graphic editor.
Unfortunately, all Maki-chan specifications I could find were in Japanese. So, as an aside on my work on SuperSakura, I tried to piece together a decent English specification. Although most games I'm working on use a refined compression algorithm and only share a Maki-like header, there are a number of images in Tenshitachi no Gogo Collection 1 that use the MAG v2 format.
Afterward, the excellent Fábio Schmidlin got in touch to help improve this specification with a better understanding of the file header and the realisation that there's more than one version of the format! Thank you very much. ^_^
Each Maki-chan file starts with a signature: MAKI01A, MAKI01B, or MAKI02, padded to 8 bytes with spaces. This indicates the format version, where MAKI02 means it's a MAG v2 file.
(MSB first, since MAKI originated on the Sharp X68000)
|0||8||Signature "MAKI01A " or "MAKI01B "|
|8||4||Computer model that the image was saved on, e.g. PC98, PC88, ESEQ, X68K, MSX2|
|12||20||User name of whoever saved the image, and other random metadata, encoded as Shift-JIS, usually has byte $1A at the end but not always|
|32||2||Size of "flag B" section, in bytes|
|34||2||Size of "pixel data A" section, in bytes|
|36||2||Size of "pixel data B" section, in bytes|
|38||2||Extension flag, see below|
|40||2||Top left corner X offset, only 00 00 allowed|
|42||2||Top left corner Y offset, only 00 00 allowed|
|44||2||Image pixel width|
|46||2||Image pixel height|
|48||48||Palette: 16 byte triplets, order GRB|
Only the two lowest bits of the extension flag are defined. None of the MKI files I have use either of these, so I can't confirm the exact effect.
Bit 0 is the 200-row flag, or aspect ratio flag. If this is clear, use a normal pixel aspect ratio. If set, all pixels are twice as tall as they are wide, so after decompressing, you may need to stretch the image vertically to make it look nice. If this bit is set, image data shouldn't exceed a height of 200 pixels – unless the computer model is an MSX, in which case expect a height of up to 212 pixels.
Bit 1 is the digital 8-color flag. You can probably ignore this. The palette is always 16 colors long regardless, with the last eight colors the same as the first eight.
When reading the palette, note that only the top 4 bits are significant. According to the specification, the bottom nibble must be set to 0 if the top is 0, otherwise it must be set to $F. So values $10..$FF must get OR $0F, while values $00..$0F must get AND $F0. (Some image loaders just copy the top nibble into the bottom; the visual difference is negligible.)
The header and palette are immediately followed by a 1000-byte "Flag A" section, then a variable-size "Flag B" section, and a variable-size pixel data section. The pixel data is nominally split in two due to the 64k segment size on 16-bit systems, but the second section immediately follows the first, so you can treat it as a single stream. The sizes given in the header may be incorrect, but happily only flag A and flag B must be accessed simultaneously. Once you're done with those, you can stream the pixel data directly from where you stopped reading flag B.
You can download the example MKI file here to test your decompressor.
You need a temporary buffer of 320x400 boolean values. (In this early image format, all pictures were assumed to be full-screen, so the 320x400 bit buffer is a constant size.) Feel free to use bytes or native integers if it's easier. This buffer is used to distinguish between areas containing vertically repeated pixels from areas of non-repeating pixels.
The temporary buffer is written into in 4x4 bit chunks. So, the first write affects buffer bits [0,0] to [3,3], the second write affects [4,0] to [7,3], and so on.
Read the flag A buffer one bit at a time. If the flag A bit is 0, set the next 4x4 bit area to all 0. If the flag A bit is 1, read 2 bytes from the flag B buffer and write those bits into the next 4x4 bit area. The top nibble of flag B byte 1 goes in the top row, the bottom nibble on the second row, the top nibble of flag B byte 2 goes on the third row, and the bottom nibble on the lowest row.
Here's an example with the first three flag A bits 0, 1, and 0, and the first two flag B bytes $8F and $67, outputting single bits into a temporary buffer:
|flag a == 0||flag a == 1||flag a == 0|
As an additional complication, the flag buffer is expected to be in MSB byte order, so you may have to swap endianness somewhere along the way. The outline image shows what your temporary buffer should look like, on the blue right side. The red left side happens if you unpack the flag buffers correctly, but didn't swap endianness afterward. (It's possible to leave the endianness as it is and cancel the distortion out in the next stage, if you're feeling clever.)
Now, set up an output buffer that can fit 640x400 4-bit pixels. Start reading the temporary buffer one bit at a time, from the top left. You may need to swap the buffer's endianness before reading.
For each bit in the temporary buffer, if the bit is 0, set the next two 4-bit pixels to zero. If the bit is 1, fetch the next pixel data byte, and save that as the next two pixels. The data byte's lower nibble is the left pixel.
The pixel indexes still need a XOR filter, but you can try to render and display the current output buffer using the proper palette. It should look like this more colorfully outlined image.
Finally, a bitwise XOR filter is applied to take care of vertical pattern repetition. This step differs between the two MAKI v1 versions.
Now you can render the final image.
(LSB first, native x86 and PC-98 form)
|0||8||Signature "MAKI02 " with two spaces at the end|
|8||4||Computer model or image editor that the image was saved on, e.g. PC98, MSX+, MPS, XPST|
|12||..||User name of whoever saved the image, and other random metadata, encoded as Shift-JIS. Variable length, terminates with byte $1A, and the first 00 after $1A marks the start of the real header|
|0||1||Start of header, always 00|
|1||1||Computer model code|
|4||2||X coordinate for image's left edge|
|6||2||Y coordinate for image's top edge|
|8||2||X coordinate for image's right edge|
|10||2||Y coordinate for image's bottom edge|
|12||4||Offset from start of header to "flag A" stream|
|16||4||Offset from start of header to "flag B" stream|
|20||4||Size of "flag B" stream, in bytes|
|24||4||Offset from start of header to "color index" stream|
|28||4||Size of "color index" stream, in bytes|
|32||..||Palette: size varies depending on screen mode, but usually 16 byte triplets, order GRB|
The edge coordinates are used to position non-fullscreen images, such as for facial expression sprites. The coordinates are inclusive, so for most images the values will be (0,0) to (639,399), indicating a 640x400 pixel image. However, during decompression, the left and right edges must be padded to coordinates that are a multiple of 4 bytes. The top and bottom are not changed.
Example: 16-color image with coordinates (283,643) to (432,789). Since 4 bits per pixel means 2 pixels per byte, you want the closest outward multiples of 8 for the X coordinates: 280 and 439. While decompressing, your image must be (280,643) to (439,789) – a pixel size of 160x147. Afterward, you can crop 3 pixels from the left, and 7 pixels from the right, to return to the exact image size.
The model code, model flags and screen mode are irrelevant for the majority of MAG files. They relate to hardware differences in the various computers used back then. For example, the PC-88 and earliest PC-98 models only had screen modes with up to 8 simultaneous colors, while the later MSX2+ could display thousands of colors but only when using a mildly terrifying color encoding method. For our purposes, MSX is the only model code that requires an additional decompression step; all the others work exactly the same.
|$00||PC-98, X68000, many others|
|$03||MSX, MSX2, MSX2+; see the end of this document|
|$68||XPST, Chironon's Paint System Tool ported to X68000 by Kenna|
|$70||MPS images for slightly newer PC-98 models|
|Screen mode||Binary screen mode||Vertical|
|0||0000 0000||400+ rows||16 colors, analog [PC-98]|
|1||0000 0001||200 rows||16 colors, analog [MSX Scr 7]|
|2||0000 0010||400 rows||8 colors, analog [VM 98]|
|3||0000 0011||200 rows||8 colors, analog [late PC-88]|
|4||0000 0100||400 rows||16 colors, digital|
|5||0000 0101||200 rows||16 colors, digital|
|6||0000 0110||400 rows||8 colors, digital [early PC-98]|
|7||0000 0111||200 rows||8 colors, digital [PC-88]|
|128||1000 0000||400+ rows||256 colors, analog [late PC-98]|
|129||1000 0001||212 rows||256 colors, analog [MSX Scr 8]|
|132||1000 0100||212 rows||256 colors, analog [MSX]|
Mode bit 0 most commonly indicates a 640x200 resolution, or in general a resolution with a 1:2 pixel aspect ratio, where all pixels are twice as tall as they are wide. After decompressing such images, you may need to stretch them to double height.
Mode bit 1 indicates an 8-color mode. As with Maki v1 images, you can ignore this. The palette is present and 16 colors long, regardless.
Mode bit 2 indicates a digital mode, and can be ignored. Some of the older PC models had a fixed 8-color palette, termed "digital" because the RGB components could only be fully on or off.
The top bit indicates a 256-color mode.
Finally, the palette always follows the header and is 16 GRB triplets long, unless the image has 256 colors, in which case the palette is 256 triplets. As with Maki v1, only the top 4 bits are significant. The bottom nibble must be set to 0 if the top is 0, otherwise it must be set to $F.
You can download the example MAG file here to test your decompressor.
The image is processed in standard left-to-right, top-down form. The compressed data is split into three streams, as delineated in the header: "Flag A", "Flag B", and a "Color index" stream. In addition, you need to keep a rolling Action buffer that can contain one pixel row's worth of "Flag B" bytes (remember, your pixel row width is padded to a multiple of 4 bytes at this point); and you need an Output image buffer, obviously.
Correct sizes for the Action and Output buffers:
The decompression algorithm is pretty straightforward. Initialise the Action buffer to all zeroes, then start processing:
How to get 16-bit values using the Action nibbles:
If the nibble is 0, read the next 16-bit value from the "Color index" stream. Otherwise, you need to copy a 16-bit value from earlier in the Output buffer at one of 15 relative locations, indicated by the Action nibble value. The official specification uses a diagram similar to this to illustrate:
As you can see, this makes use of the repetition in ordered dithering patterns. 0 is your current location in the Output buffer. 1 indicates you should copy the previous 16-bit value; 2 indicates the value two steps behind the current position; 3, four steps back; 4, the value one row above the current position; 5, one row above and one step back. And so on.
Example: 640x400 pixel image, 4 bits per pixel, each row is
320 bytes wide.
The current output offset is $8A0, and the action byte is $05.
Action byte top nibble: at offset $8A0, write the next 16-bit value from the "Color index" stream.
Bottom nibble: at offset $8A2, copy the 16-bit value from the Output buffer at offset $8A2 - 2 - 320.
Here are the delta values put into arrays for your convenience:
You can download the example MAX file here to test your decompressor.
The MSX computers supported common 16-color resolutions, as well as some more exotic ones. Some image file names have a .MAX ending, but internally they are identical to MAG v2. Most MAG v2 images tagged with the MSX model code can be decompressed normally as described above.
|3||$00||Normal 16-color image|
|3||$04||Normal, probably uses pixel ratio 1:2|
|3||$14||Normal 256x212 image, pixel ratio 1:1, 256 colors?|
|3||$24||MSX screen mode 10|
|3||$34||MSX screen mode 11|
|3||$44||MSX screen mode 12|
|3||$54||Normal 256x212 image, pixel ratio 1:1, 16 colors?|
This is where the model-dependent flag comes in. If the image uses one of the extended screen modes ($24, $34, $44), it'll need an extra step after decompression. First, unpack it normally to a 4-bit indexed form at the size specified in the header. Don't stretch to double height even if screen mode bit 0 is set in the header.
The bitmap should look like a colorful mess with distinct vertical lines, as in the below image.
The colors are YJK-encoded, and must be converted to RGB, which also halves the image's real pixel width. J and K are 6-bit signed values, shared between four neighboring pixels, each of which has a 5-bit Y value. The output RGB values are also 5-bit, so when you're done, you'll need to scale those to a more standard 8-bit depth (multiply by 255 and divide by 31).
Here's how to convert:
Example: Model flag $44, YJK bytes 87 8F 8B 81, or in binary:
K = 111 111 = -1
J = 001 011 = 11
Yn = 16, 17, 17, 16
1st output 5-bit pixel: R 27, G 15, B 14.75
2nd output 5-bit pixel: R 28, G 16, B 16
3rd output 5-bit pixel: R 28, G 16, B 16
4th output 5-bit pixel: R 27, G 15, B 14.75
If done correctly, you'll end up with a tiny but beautifully-colored image like this one.