m64 format documentation
#1
Through reverse-engineering of the SM64 audio library, I figured some more details of the m64 format.

The hack64 forum software doesn't seem to support tables, but here's an external markdown doc: https://hackmd.io/opEB-OmxRa26P8h8pA-x7w as well as a decoder: https://gist.github.com/simonlindholm/30...22c291f878

The list of commands should be complete, though some of the descriptions are a bit vague. I could look closer into the code if there's some particular aspect that people are interested in.
jesusyoshi54, lezg_g, Nutta, and Sauraen liked this post
Reply
#2
(07-16-2019, 02:33 PM)simonlindholm Wrote: Through reverse-engineering of the SM64 audio library, I figured some more details of the m64 format.

The hack64 forum software doesn't seem to support tables, but here's an external markdown doc: https://hackmd.io/opEB-OmxRa26P8h8pA-x7w as well as a decoder: https://gist.github.com/simonlindholm/30...22c291f878

The list of commands should be complete, though some of the descriptions are a bit vague. I could look closer into the code if there's some particular aspect that people are interested in.

Man, thank you! this helps a lot! I added some commands in seq64 and works really great.  Cool
Reply
#3
(07-16-2019, 02:33 PM)simonlindholm Wrote: Through reverse-engineering of the SM64 audio library, I figured some more details of the m64 format.

The hack64 forum software doesn't seem to support tables, but here's an external markdown doc: https://hackmd.io/opEB-OmxRa26P8h8pA-x7w as well as a decoder: https://gist.github.com/simonlindholm/30...22c291f878

The list of commands should be complete, though some of the descriptions are a bit vague. I could look closer into the code if there's some particular aspect that people are interested in.

Awesome work simonlindholm! How did you figure the rest of the commands out? Just by reading the disassembly? There are a lot of data structures used by the sequence parsing, and even with stepping through the code in an emulator it is very difficult to figure out what all the fields would be.

I'm the author of SEQ64, which is software for importing/exporting this sequence file format from/to MIDI, both in SM64 and the other first-party Nintendo games which use versions of the same file format (most notably OoT and MM). SEQ64 is also the backend for N64 Sound Tools for the games using these formats. I figured out many of these commands years ago, starting from previous work by Deathbasket, when I was developing the program, but I was never able to figure out the rest of them. I hope you were able to build on my work--I see you're using my notation of "Q" for the accumulator, so I'm hoping you were able to start from the sequence definition in SEQ64 and figure out the rest from there.

The reason SEQ64 is 20,000 lines of code rather than a few hundred is because it allows the definition of the sequence format to be specified by the user, rather than being hard-coded into the program. This was primarily intended to avoid minor differences between games causing the program to only be usable for certain games or versions, because nobody wants to change the code and recompile it. But that does mean it became large and unwieldy in some ways, and it was never able to use the scripting capabilities of the format besides optimizing data with calls and loops. It would be nice to have a full script editor that's complementary to SEQ64, where you could import a MIDI with SEQ64 and then edit it as a script with the other tool. Of course, this tool should also work on OoT and all other games using similar formats, which is a lot to ask!
Sauraen#0047 / SEQ64 developer: https://github.com/sauraen/seq64
Reply
#4
Great job on this though there's some little things you should change, the 0xFD command doesn't actually delay based on frames, it delays based on ticks though they might work the same way.


Code:
32nd note - 12 ticks 0x0c

16th note - 24 ticks 0x18

8th note - 48 ticks 0x30

4th note - 72 ticks 0x48

2nd note - 96 ticks 0x60

Whole note = 192 ticks 0xc0

4 bars =  0x300
Reply
#5
(08-16-2019, 05:57 PM)Sauraen Wrote:
(07-16-2019, 02:33 PM)simonlindholm Wrote: Through reverse-engineering of the SM64 audio library, I figured some more details of the m64 format.

The hack64 forum software doesn't seem to support tables, but here's an external markdown doc: https://hackmd.io/opEB-OmxRa26P8h8pA-x7w as well as a decoder: https://gist.github.com/simonlindholm/30...22c291f878

The list of commands should be complete, though some of the descriptions are a bit vague. I could look closer into the code if there's some particular aspect that people are interested in.

Awesome work simonlindholm! How did you figure the rest of the commands out? Just by reading the disassembly? There are a lot of data structures used by the sequence parsing, and even with stepping through the code in an emulator it is very difficult to figure out what all the fields would be.
By reading the disassembly, yeah. (Helped by decompilation into C using https://github.com/matt-kempster/mips_to_c.) The SM64 audio lib indeed has a lot of interconnected structures, so it was non-trivial...

Quote:I'm the author of SEQ64, which is software for importing/exporting this sequence file format from/to MIDI, both in SM64 and the other first-party Nintendo games which use versions of the same file format (most notably OoT and MM). SEQ64 is also the backend for N64 Sound Tools for the games using these formats. I figured out many of these commands years ago, starting from previous work by Deathbasket, when I was developing the program, but I was never able to figure out the rest of them. I hope you were able to build on my work--I see you're using my notation of "Q" for the accumulator, so I'm hoping you were able to start from the sequence definition in SEQ64 and figure out the rest from there.

I definitely used SEQ64 to help understand some of the structures involved and the meaning of various commands in non-code terms, and to verify the sanity of other parts. The "Q" notation was to align with SEQ64, though I actually found out about SEQ64 only after I got the Q-related commands down.

Quote:The reason SEQ64 is 20,000 lines of code rather than a few hundred is because it allows the definition of the sequence format to be specified by the user, rather than being hard-coded into the program. This was primarily intended to avoid minor differences between games causing the program to only be usable for certain games or versions, because nobody wants to change the code and recompile it. But that does mean it became large and unwieldy in some ways, and it was never able to use the scripting capabilities of the format besides optimizing data with calls and loops. It would be nice to have a full script editor that's complementary to SEQ64, where you could import a MIDI with SEQ64 and then edit it as a script with the other tool. Of course, this tool should also work on OoT and all other games using similar formats, which is a lot to ask!

Yeah, the flexible design of SEQ64 certainly makes a lot of sense, and it's always convenient to have things as data instead code in general. I agree that a good standalone script editor would be neat.

Re scripting capabilities, one of the fun parts of having this documentation was to see the tricks they used for sound effects in SM64, in sequence 0. For instance, here's how a red coin sound gets played:

- the game calls SetSound(0x78289081 + (coin number << 16)), which writes 0x28 + coin number to IO slot 4 of channel 7 of sequence 0, and 1 ("play") to IO slot 0.
- the write to slot 0 causes the script for channel 7 to break out of the following loop:
Code:
...
.chan_FD:
chan_delay1
chan_ioreadval 0
chan_bltz .chan_FD
and do
Code:
chan_setdyntable .table_2B1D
chan_ioreadval 4
chan_dyncall
- at 0x2B1D in the sequence is a table of data, which at positions 0x28..0x2F all contain 0x3122. At location 0x3122 is the following channel script:
Code:
.chan_3122:
chan_setinstr 128
chan_setnotepriority 14
chan_setpanchanweight 0
chan_setenvelope .envelope_3378
chan_ioreadval 4
chan_subtract 0x28
chan_readseq .addr_313E
chan_writeseq 0, .layer_fn_3188, 1
chan_setlayer 0, .layer_3146
chan_setlayer 1, .layer_3168
chan_setlayer 2, .layer_3148
chan_end

.addr_313E:
.byte 0x0
.byte 0x2
.byte 0x4
.byte 0x5
.byte 0x7
.byte 0x9
.byte 0xb
.byte 0xc

.layer_3146:
layer_delay 0x6

.layer_3148:
layer_call .layer_fn_3188
layer_note0 46, 0xc, 75, 20
layer_note0 45, 0xc, 75, 20
layer_note0 46, 0xc, 75, 20
layer_note0 58, 0x10, 80, 80
layer_note0 58, 0x10, 45, 80
layer_note0 58, 0x10, 20, 80
layer_note0 58, 0x10, 15, 80
layer_end

.layer_3168:
layer_call .layer_fn_3188
layer_note0 41, 0xc, 75, 20
layer_note0 40, 0xc, 75, 20
layer_note0 41, 0xc, 75, 20
layer_note0 53, 0x10, 80, 80
layer_note0 53, 0x10, 45, 80
layer_note0 53, 0x10, 20, 80
layer_note0 53, 0x10, 15, 80
layer_end

.layer_fn_3188:
layer_transpose 0
layer_end
which reads IO slot 4, subtracts 0x28, maps it to the major scale using a lookup table, overwrites the "transpose" command of a function in the sequence script, and calls that function from the channel layers before finally playing the coin sound, now with a pitch corresponding to the index of the red coin. I found that pretty neat. :)

(08-17-2019, 01:00 PM)Nutta Wrote: Great job on this though there's some little things you should change, the 0xFD command doesn't actually delay based on frames, it delays based on ticks though they might work the same way.
Thanks. Ticks vs. frames was actually a typo, fixed.
Reply
#6
simonlindholm Wrote:overwrites the "transpose" command of a function in the sequence script

So not only does the sequence format contain a Turing-complete scripting language, it uses self-modifying code? :O
Sauraen#0047 / SEQ64 developer: https://github.com/sauraen/seq64
Reply
#7
Yep. :) Makes a little bit of sense, so they didn't have to implement dynamic versions of all commands, though it's also somewhat terrifying. You could also use the sequence read/write commands to implement memory, making sequence scripts even more clearly Turing complete.
Reply
#8
Have you checked what happens if the sequence write command attempts to write outside the sequence itself? If that isn't checked and prohibited, and somehow arbitrary sequence execution was obtained, that could be used to obtain arbitrary code execution...
Sauraen#0047 / SEQ64 developer: https://github.com/sauraen/seq64
Reply
#9
No checks against it, so it could totally aid in arbitrary code execution if you were able to manipulate sequence data somehow. Though it's limited to a 2^16-byte range, and in SM64's case there's nothing interesting after the sequence data, so you would have to get creative with other unchecked commands as well.
Reply
#10
There's nothing interesting in the entire almost-64 kB after the sequence data? Really? Have you checked whether the address value is sign-extended, so an address of e.g. FFFF accesses memory before the sequence? If the memory allocation works similarly to malloc, writing to somewhere around FFF8 or FFFC could corrupt the heap after the sequence is freed.
Sauraen#0047 / SEQ64 developer: https://github.com/sauraen/seq64
Reply




Users browsing this thread: 2 Guest(s)