m64 format documentation

[simonlindholm]

Through reverse-engineering of the SM64 audio library, I figured some more details of the m64 format.

The hack64 forum software doesn't seem to support tables, but here's an external markdown doc: https://hackmd.io/opEB-OmxRa26P8h8pA-x7w as well as a decoder: https://gist.github.com/simonlindholm/30...22c291f878

The list of commands should be complete, though some of the descriptions are a bit vague. I could look closer into the code if there's some particular aspect that people are interested in.

Awesome work simonlindholm! How did you figure the rest of the commands out? Just by reading the disassembly? There are a lot of data structures used by the sequence parsing, and even with stepping through the code in an emulator it is very difficult to figure out what all the fields would be.

By reading the disassembly, yeah. (Helped by decompilation into C using https://github.com/matt-kempster/mips_to_c.) The SM64 audio lib indeed has a lot of interconnected structures, so it was non-trivial...

I'm the author of SEQ64, which is software for importing/exporting this sequence file format from/to MIDI, both in SM64 and the other first-party Nintendo games which use versions of the same file format (most notably OoT and MM). SEQ64 is also the backend for N64 Sound Tools for the games using these formats. I figured out many of these commands years ago, starting from previous work by Deathbasket, when I was developing the program, but I was never able to figure out the rest of them. I hope you were able to build on my work--I see you're using my notation of "Q" for the accumulator, so I'm hoping you were able to start from the sequence definition in SEQ64 and figure out the rest from there.

I definitely used SEQ64 to help understand some of the structures involved and the meaning of various commands in non-code terms, and to verify the sanity of other parts. The "Q" notation was to align with SEQ64, though I actually found out about SEQ64 only after I got the Q-related commands down.

The reason SEQ64 is 20,000 lines of code rather than a few hundred is because it allows the definition of the sequence format to be specified by the user, rather than being hard-coded into the program. This was primarily intended to avoid minor differences between games causing the program to only be usable for certain games or versions, because nobody wants to change the code and recompile it. But that does mean it became large and unwieldy in some ways, and it was never able to use the scripting capabilities of the format besides optimizing data with calls and loops. It would be nice to have a full script editor that's complementary to SEQ64, where you could import a MIDI with SEQ64 and then edit it as a script with the other tool. Of course, this tool should also work on OoT and all other games using similar formats, which is a lot to ask!

Yeah, the flexible design of SEQ64 certainly makes a lot of sense, and it's always convenient to have things as data instead code in general. I agree that a good standalone script editor would be neat.

Re scripting capabilities, one of the fun parts of having this documentation was to see the tricks they used for sound effects in SM64, in sequence 0. For instance, here's how a red coin sound gets played:

- the game calls SetSound(0x78289081 + (coin number << 16)), which writes 0x28 + coin number to IO slot 4 of channel 7 of sequence 0, and 1 ("play") to IO slot 0.
- the write to slot 0 causes the script for channel 7 to break out of the following loop:

Code:

...
.chan_FD:
chan_delay1
chan_ioreadval 0
chan_bltz .chan_FD

and do

Code:

chan_setdyntable .table_2B1D
chan_ioreadval 4
chan_dyncall

- at 0x2B1D in the sequence is a table of data, which at positions 0x28..0x2F all contain 0x3122. At location 0x3122 is the following channel script:

Code:

.chan_3122:
chan_setinstr 128
chan_setnotepriority 14
chan_setpanchanweight 0
chan_setenvelope .envelope_3378
chan_ioreadval 4
chan_subtract 0x28
chan_readseq .addr_313E
chan_writeseq 0, .layer_fn_3188, 1
chan_setlayer 0, .layer_3146
chan_setlayer 1, .layer_3168
chan_setlayer 2, .layer_3148
chan_end

.addr_313E:
.byte 0x0
.byte 0x2
.byte 0x4
.byte 0x5
.byte 0x7
.byte 0x9
.byte 0xb
.byte 0xc

.layer_3146:
layer_delay 0x6

.layer_3148:
layer_call .layer_fn_3188
layer_note0 46, 0xc, 75, 20
layer_note0 45, 0xc, 75, 20
layer_note0 46, 0xc, 75, 20
layer_note0 58, 0x10, 80, 80
layer_note0 58, 0x10, 45, 80
layer_note0 58, 0x10, 20, 80
layer_note0 58, 0x10, 15, 80
layer_end

.layer_3168:
layer_call .layer_fn_3188
layer_note0 41, 0xc, 75, 20
layer_note0 40, 0xc, 75, 20
layer_note0 41, 0xc, 75, 20
layer_note0 53, 0x10, 80, 80
layer_note0 53, 0x10, 45, 80
layer_note0 53, 0x10, 20, 80
layer_note0 53, 0x10, 15, 80
layer_end

.layer_fn_3188:
layer_transpose 0
layer_end

which reads IO slot 4, subtracts 0x28, maps it to the major scale using a lookup table, overwrites the "transpose" command of a function in the sequence script, and calls that function from the channel layers before finally playing the coin sound, now with a pitch corresponding to the index of the red coin. I found that pretty neat. :)

Great job on this though there's some little things you should change, the 0xFD command doesn't actually delay based on frames, it delays based on ticks though they might work the same way.

Thanks. Ticks vs. frames was actually a typo, fixed.