|MAG errata||MAG errata|
Various Japanese specifications and native X68000 software can be found here. The table on the right links to the specifications.
Japanese history of the MAG format at a doujin dictionary: http://www.paradisearmy.com/doujin/pasok4u.htm
You can find a ton of images archived as the MSX Pixel Art Preservation Collection, over here.
Another browser-based MAG viewer is here. Just drag and drop a picture file in the browser window! It'll only open MAG v2 files, not MAKI v1 files.
My own code for opening v1 and v2 files is on Gitlab as part of SuperSakura. You can convert MKI, MAG, and MAX files to PNG with the "decomp" tool. There is a win32 binary of the whole project here, which contains decomp.
Back in the day before LZW compression became ubiquitous, less efficient image compression methods were used. Woody Rinn's MAKI format (or, more familiarly, Maki-chan graphics) was an early storage format in Japan. He developed the refined MAG v2 format soon after, which became the dominant sharable image type until the World Wide Web brought GIFs into popular consciousness. Rinn-san is actually a capable illustrator as well as programmer, and has a few commercially published mangas. He developed the Maguro Paint System (MPS) (later rebranded to Multi Paint), which became the most popular PC-98 graphic editor.
Unfortunately, all Maki-chan specifications I could find were in Japanese. So, as an aside on my work on SuperSakura, I tried to put together a decent English specification. Although most games I was trying to support use the more advanced Pi format, there are a handful of MAG v2 images in Tenshitachi no Gogo Collection 1.
Afterward, the excellent Fábio Schmidlin got in touch to help improve this specification with a better understanding of the file header and the realisation that there's more than one version of the format! Thank you very much. ^_^
Each Maki-chan file starts with a signature: MAKI01A, MAKI01B, or MAKI02, padded to 8 bytes with spaces. This indicates the format version: MAKI v1 or MAG v2.
MAKI v1 header
(MSB-first, since MAKI originated on the Sharp X68000)
|0||8||Signature "MAKI01A " or "MAKI01B "|
|8||4||Computer model that the image was saved on, e.g. PC98, PC88, ESEQ, X68K, MSX2|
|12||20||User name etc. metadata string, encoded in Shift-JIS; usually terminates with byte $1A, but not always|
|32||2||Size of "flag B" section, in bytes|
|34||2||Size of "pixel data A" section, in bytes|
|36||2||Size of "pixel data B" section, in bytes|
|38||2||Extension flag, see below|
|40||2||Top left corner X offset, only 00 00 allowed|
|42||2||Top left corner Y offset, only 00 00 allowed|
|44||2||Image pixel width|
|46||2||Image pixel height|
|48||48||Palette: 16 byte triplets, order GRB|
(Note: The user name string may contain some MSX-specific screen mode values. Please see the errata linked at the top of the document for details. However, I haven't found any MAKI v1 images actually using this.)
The extension flag, as defined, only uses the lowest two bits:
Bit 0 is the 200-row flag, or aspect ratio flag. If this bit is not set, use a normal pixel aspect ratio. If set, all pixels are twice as tall as they are wide; after decompressing normally, stretch the image to double height to unsquash it.
Bit 1 is the digital 8-color flag, used to indicate a legacy screenmode with only 3 bits per pixel. I have no example images using this, so I can't verify how exactly it works.
The image width and height can probably be ignored. The MAKI v1 algorithm assumes that all images are full-screen 640x400. Larger images are not allowed as defined, and smaller images are padded to full-screen with solid borders. For this reason, the top left corner offset is not really used either.
When reading the palette, note that only the top 4 bits are significant. By the specification, the bottom nibble must be set to 0 if the top is 0; otherwise, it must be set to $F. So values $10..$FF must get a bitwise OR $0F, and values $00..$0F must get a bitwise AND $F0. (I think for PC98 images specifically, where the palette was limited by hardware to 4 bits per channel, it would be more accurate to copy the top nibble into the bottom, but the visual difference is negligible.)
MAKI v1 data layout
|96||1000||Flag A section|
|1096||...||Flag B section|
|...||...||Pixel data section|
The header and palette are immediately followed by a 1000-byte "flag A" section, then a variable-size "flag B" section, and a variable-size pixel data section.
The pixel data is nominally split in two sections due to the 64k segment size on 16-bit systems, but the second section immediately follows the first, so you can treat it as a single block.
The section sizes given in the header may be incorrect, but this is not a problem. During the first decompression phase, only the A and B sections are being read. Pixel data is applied during the second phase, starting from the next byte after the last flag B byte you processed.
MAKI v1 decompression
Click the image to download the example MKI file, to test your decompressor.
First, set up a temporary buffer. This is for an image mask marking pixels that don't repeat vertically. Because all pictures are assumed to be full-screen, the buffer is always 320x400. Feel free to make it an array or table of 128000 booleans, bytes, or native integers, whatever's easiest.
The temporary buffer is written into in 4x4 chunks. So, the first write goes in indexes [0,0] to [3,3], the second write in [4,0] to [7,3], and so on.
Start reading the flag A section one bit at a time, topmost bit first. (So start from bitmask $80, then $40, then $20...)
If the flag A bit is 0, set the next 4x4 temporary buffer chunk to all 0.
If the flag A bit is not 0, read 2 bytes from the flag B section and use them for the next 4x4 temporary buffer chunk.
|flag A (binary)|
|flag B (hexadecimal)|
|first output 4x4|
(a == 0)
|second output 4x4|
(a == 1)
|third output 4x4|
(a == 0)
As an additional complication, the temporary buffer is expected to be in MSB-first byte order, so you may have to swap endianness somewhere along the way. The outline image shows what your temporary buffer should look like, on the blue right side. The red left side happens if you unpack the flag buffers correctly, but didn't swap endianness afterward. (It's possible to leave the endianness as is and cancel the distortion out in the next stage, if you're feeling clever.)
Next, set up a 128000-byte output buffer. It will take 640x400 4-bit palette indexes.
Start reading your temporary buffer one value at a time, from the top left. You may need to swap the buffer's endianness while reading.
If the value is 0, output a 0 byte.
If the value is not 0, fetch the next pixel data byte, and output that.
|pixel data (hexadecimal)|
(temp == 0)
(temp == 1)
(temp == 1)
(temp == 0)
At this point you can try to render and display the current output buffer using the image's proper palette. (Each byte has two palette indexes; the lower nibble is the left pixel in the pair.) It should look like this more colorfully outlined image.
Finally, apply a bitwise XOR filter to take care of vertical pattern repetition. This step differs between the two MAKI v1 versions:
- MAKI01A: From the start of the 3rd row to the end of the image, XOR all bytes with the bytes from TWO rows above.
- MAKI01B: From the start of the 5th row to the end of the image, XOR all bytes with the bytes from FOUR rows above.
Now you can render the final image.
MAG v2 header
(LSB-first, native x86 and PC-98 form)
|0||8||Signature "MAKI02 " with two spaces at the end|
|8||4||Computer model or image editor that the image was saved on, e.g. PC98, MSX+, MPS, XPST|
|12||..||User name etc. metadata string, encoded in Shift-JIS; Variable-length, terminates with byte $1A, and the first 00 after $1A marks the start of the real header|
|0||1||Start of header, always 00|
|1||1||Computer model code|
|4||2||X coordinate for image's left edge|
|6||2||Y coordinate for image's top edge|
|8||2||X coordinate for image's right edge|
|10||2||Y coordinate for image's bottom edge|
|12||4||Offset from start of header to "flag A" stream|
|16||4||Offset from start of header to "flag B" stream|
|20||4||Size of "flag B" stream, in bytes|
|24||4||Offset from start of header to "color index" stream|
|28||4||Size of "color index" stream, in bytes|
|32||..||Palette: up to 256 byte triplets, order GRB|
The model code, model-dependent flags, and screen mode mostly relate to hardware differences in the various computers used back then. For example, the PC-88 and earliest PC-98 models only had screen modes with up to 8 simultaneous colors. The later MSX2+ could display thousands of colors but only by using a mildly terrifying color encoding scheme. However, the basic decompression steps are always the same regardless of hardware, and images can be much larger than the screen mode resolution.
|$00||PC-98, X68000, many others|
|$03||MSX, MSX2, MSX2+; requires special handling|
|$68||X68K or XPST, Chironon's Paint System Tool ported to X68000 by Kenna|
|$70||MPS images for slightly newer PC-98 models|
The model code slightly affects how the image palette is interpreted. If the model code is $03, the model-dependent flag byte overrides the screen mode byte. Otherwise the flag byte can be ignored.
|Model code||Flag||Colors||Pixel ratio|
|$03||$00||16||1:1||MSX2 screen mode 7, 512x212|
|$03||$04||16||1:2||MSX2 screen mode 7, 512x212|
|$03||$14||256||1:1||MSX2 screen mode 8, 256x212|
|$03||$24||12499+||1:1||MSX2+ screen mode 10, 256x212|
|$03||$34||12499+||1:1||MSX2+ screen mode 11, 256x212|
|$03||$44||19268||1:1||MSX2+ screen mode 12, 256x212|
|$03||$54||16||1:1||MSX2 screen mode 5, 256x212|
|3||200 rows||8||1:2||late PC-88|
|6||400 rows||8||1:1||early PC-98|
|128||400+ rows||256||1:1||late PC-98|
|129||212 rows||256||hardware palette?|
|133||212 rows||256||hardware palette?|
The screen mode's top bit indicates a 256-color mode, or 8 bits per pixel. When the top bit is set, the bottom bit may indicate that the image uses a special hardware palette, but so far all my test images have embedded normal palette data anyway.
When the top bit is not set, the bottom bit indicates a 1:2 pixel aspect ratio, where all pixels are twice as tall as they are wide. Such images need to be stretched to double height as the final step after decompressing, or they'll look squashed.
You can ignore mode bit 1, which indicates an 8-color mode. The palette is still present like on any other image.
You can ignore mode bit 2, which indicates a digital mode. Some of the older PC models had a fixed 8-color palette, termed "digital" because the RGB components could only be fully on or off.
The edge coordinates are used to position non-fullscreen images, such as for facial expression sprites. The coordinates are inclusive; for most images the values will be (0,0) to (639,399), indicating a 640x400 pixel image.
The compressed image is split into three streams, as delineated in the header: "flag A", "flag B", and a "color index" stream. You can ignore the stream sizes listed in the header. Flag A runs up to the start of the flag B stream; flag B runs up to the start of the color stream; the color stream runs to the end of the file.
Finally, the palette is at the end of the header and can be up to 256 GRB triplets long. For best compatibility, initialise a 256-color palette structure, then keep reading GRB triplets up to the first byte of the flag A stream.
Although the palette color components are always one byte each, they have a different number of significant bits depending on the computer model. Use these guidelines to determine the number of significant bits:
- By default, all images use 4 bits
- If model code is $03, use 3 bits (the MSX palette can do 512 colors)
- Except if the file contains the string "Deca loader" at offset 32, use 4 or 8 bits (it's an image viewer that somehow displayed higher bitdepths than the MSX palette allowed)
- If model code is $68, use 5 bits (the X68k has 16-bit graphics, but the intensity bit is ignored for palettisation)
- If model code is $99, use 8 bits (the Mac2 has 24-bit graphics)
- If the palette is 256 colors long, and model code is not $03 or $88, use 8 bits
Every palette component byte should have its most significant bits copied to the rest of the byte. Examples:
|Input byte||Significant bits||Input binary, masked||Final binary||Final byte|
|$55||3||010 xxx xx||010 010 01||$49|
|$BF||4||1011 xxxx||1011 1011||$BB|
|$67||5||01100 xxx||01100 011||$63|
MAG v2 decompression
Click the image to download the example MAG file, to test your decompressor.
Before anything else, the left and right image edges must be padded to the next nearest multiple of 4 bytes. (The top and bottom are never padded.) If the header's screen mode byte has its top bit set (the 256-color flag), bits per pixel is 8, else bits per pixel is 4. All fractions round down.
- pixels per byte = 8 / bits per pixel
- padded left edge = (header left edge / pixels per byte) AND $FFFC
- padded right edge = (header right edge / pixels per byte + 4) AND $FFFC
- padded image byte width = padded right edge - padded left edge
- padded image pixel width = padded image byte width * pixels per byte
- image pixel height = header bottom edge - header top edge + 1
Example: 16-color image with coordinates (283,643) to (432,789). This is an image of exactly 150x147 pixels. With padding, the edge byte coordinates become (140,643) to (220,789) – an image size of 80x147 bytes, or 160x147 pixels. After decompression is finished, you can crop 3 pixel columns from the left, and 7 pixel columns from the right, to get the exact image size.
To decompress, two buffers are needed: an output buffer, and an action buffer. The buffer sizes are:
- Output buffer = padded image byte width * image pixel height
- Action buffer = padded image byte width / 4
These are the purposes of the streams and buffers:
- "Flag A" is a stream of single-bit boolean flags, read one bit at a time, from highest to lowest bit in each byte. These indicate whether to fetch the next "flag B" byte or not.
- "Flag B" is an array of nibbles (4-bit values), read one byte at a time, and processed top nibble first. These are XORred into the action buffer.
- The action buffer is a row of nibbles, handled in pairs. These indicate whether to copy a 16-bit value from earlier in the output buffer, or to fetch a new value from the "color index" stream. While working, loop back to the buffer's beginning when you reach its end.
- The "color index" stream is an array of 16-bit values. You can think of these as palette index pairs or quartets.
- The output buffer is a simple byte array, although written to 2 bytes at a time.
The decompression algorithm is pretty straightforward. Initialise the action buffer to all zeroes, then start processing:
- Read the next "flag A" bit.
- If the "flag A" bit was set, then read the next "flag B" byte and XOR the next value in the action buffer with it; else do nothing.
- Read the next action buffer byte (the one you possibly just XORred).
- For the top nibble of the action byte:
- If the nibble = 0, read the next 16-bit value from the "color index" stream, and output that.
- Else copy a 16-bit value from earlier in the output buffer, as described below.
- Do the same for the bottom nibble of the action byte.
Repeat these steps until the output buffer is full, or one of the input flag streams runs out. If the "color index" stream runs out, try using zeroes for any extra color indexes until the output buffer is full.
When the action nibble is not 0, you need to copy a 16-bit value from one of 15 relative locations in the output buffer. The official specification uses a diagram similar to this to illustrate:
As you can see, this makes use of the repetition in ordered dithering patterns. 0 is the current location in the Output buffer. 1 indicates you should copy the previous 16-bit value; 2 indicates the value two steps behind the current position; 3, four steps back; 4, the value one row above the current position; 5, one row above and one step back. And so on.
Example: 320x400 byte image, current output offset $8A0.
- Action byte = $05
- Top action nibble: at offset $8A0, write the next 16-bit value from the "color index" stream.
- Bottom action nibble: at offset $8A2, copy the 16-bit value from offset $8A2 - 2 - 320.
Now you have a full output buffer, but we're not quite done yet.
If this image used an advanced MSX2+ screen mode, the YJK colors must be decoded at this point. See below.
If the output buffer is still 4 bits per pixel, it is now safe to expand it to a more natural 8 bits per pixel, or unpack the palette indexing to a truecolor image, as you prefer.
Any padding added to the left and right edges can be removed.
Finally, the image must be stretched to double height, only if:
- Model code is not $03, and the screen mode top bit is not set, and the screen mode bottom bit is set; or,
- Model code is $03, and model-dependent flag byte is 4.
MSX images and YJK conversion
You can download the example MAX file here to test your decompressor.
.MAX files are internally identical to MAG v2, and many MSX images use the .MAG extension. Images specifically intended for the MSX will have the model code set to $03.
MSX computers supported medium-resolution 16-color modes, as well as some more exotic modes using YJK encoding. Most images use the plain 16-color modes, and don't need additional steps.
This is where the model-dependent flag byte comes in. An extra step after decompression is needed only if the flag byte is $24, $34, or $44, indicating one of the YJK modes.
First, unpack the image normally, as described above. Without YJK decoding, the bitmap should look like a colorful mess with distinct vertical lines, as in the below image.
The colors must be converted to RGB. If the image had 4 bits per pixel, the conversion halves its pixel width.
In YJK, pixels are treated in groups of 4. Each pixel has its own Y value, but the J and K components are shared by all 4 pixels. Y values are unsigned 5-bit, while J and K are signed 6-bit. This allows packing four nearly high-color pixels in only four bytes.
After conversion, the output RGB values are also unsigned 5-bit, so they'll need to be scaled to a more standard 8-bit depth (multiply by 255 and divide by 31).
Here's how to convert:
- Reserve width * height * 3 bytes for the final RGB image
- Read the YJK image 4 bytes at a time
- K = low 3 bits of first byte + 8 * (low 3 bits of second byte)
- J = low 3 bits of third byte + 8 * (low 3 bits of fourth byte)
- Y0 = (high 5 bits of first byte) / 8
- Y1 = (high 5 bits of second byte) / 8
- Y2 = (high 5 bits of third byte) / 8
- Y3 = (high 5 bits of fourth byte) / 8
- For each Yn value:
- If the model flag is $24 or $34 and the lowest bit of Yn is set:
output the RGB values of palette index (Yn / 2)
- Else output these RGB values:
R = Yn + J
G = Yn + K
B = 5 * Yn / 4 - J / 2 - K / 4
- If the model flag is $24 or $34 and the lowest bit of Yn is set:
- Scale and clamp the RGB values into an acceptable range.
Model flag $44, YJK bytes 87 8F 8B 81, or in binary:
|10000||10001||10001||10000||111 111||001 011|
|Output 5-bit pixels|
Model flag $24, YJK bytes 6A 68 29 31, or in binary:
|01101||01101||00101||00110||000 010||001 001|
|Output 5-bit pixels|
|Palette index 6|
|Palette index 6|
|Palette index 2|
If converted correctly, the end result is a tiny but beautifully-colored image like this one.