SM64 Fast3D Anti-Aliasing Reducer

[David]

This is a simple tool you can use to improve performance on real N64 hardware by forcing the game to use the reduced anti-aliasing mode instead of the full anti-aliasing mode.

---

How does it work?

This tool will inject a modified version of the Fast3D 2.0D microcode that will block any attempt to write the IM_RD render mode flag into the RDP's othermode variable. Doing this forces the game to use the reduced anti-aliasing mode, which should improve performance on console by 3-4 fps (~10%).

---

How do I use it?

Just simply open up a Super Mario 64 ROM file, and then you can click the "Reduce the AA!" button if the ROM is compatible. Once the button is pressed, it will create a new ROM file with the ".ra" prefixed to the extension. For example, using the rom file "sm64.z64" will generate a new rom called "sm64.ra.z64".

---

What SM64 ROMs are supported?

Any SM64 ROM that uses the Fast3D 2.0D microcode is compatible. This includes the Japanese and North American versions of SM64 and any ROM hacks made with those versions.

---

What about emulators?

The compatibility with emulators will mainly depend on what kind of plugin that you use. In theory, high-level emulation plugins like should not have any problems if they assume that this microcode is just regular Fast3D. However, as mentioned in the replies below, some plugins like GlideN64 v4.0 doesn't seem to emulate the microcode correctly.

Also some low-level emulation plugins like angrylion do not like the custom microcode, and may make the game lag even more than before. Which is weird, because the microcode runs fine on real hardware. ¯\_(ツ)_/¯

---

Benchmark

The beginning area of Jolly Roger Bay does lag a significant amount. Going down to a 22 fps minimum in my testing. Injecting the modified microcode makes a significant difference in the feel of the game, with the minimum increased to 25 fps.

My jolly roger bay benchmark starts at the beginning of the level and has Mario swim into the underwater cave. The framerate is bad in the beginning, but gets a lot better when you are deep underwater.

---

Download & Source

Download: https://github.com/DavidSM64/SM64-Fast3D...r/releases
Source: https://github.com/DavidSM64/SM64-Fast3D-AA-Reducer/

---

[PyroJay4]

Nobody's made any replies so i'll say it first but i love you for making this

[triclon]

Quick question, is this different from the "disable AA" gameshark codes? I know those also improve performance so if this is different, yo could combine both for an ultra-high FPS experience.

[David]

Quick question, is this different from the "disable AA" gameshark codes?  I know those also improve performance so if this is different, yo could combine both for an ultra-high FPS experience.

I know people use the gameshark code to make the game look "better" by removing the AA, but I haven't seen any proof that it improves performance. As far as I know, most of the lag comes from the calculations on the RDP. 

I haven't actually tested the gameshark code myself, but both methods are different. The gameshark code disables the anti-aliasing from the Video Interface (VI), whereas this tool disables the IM_RD flag which is used by the blender in the RDP.

From the N64 programming manual:

In the blender, the pixel color and the last value stored for the pixel in memory are combined. The blender also combines the pixel coverage and memory coverage and does z-buffering. The blender typically performs operations such as antialiasing the interior edges of objects and transparency. The new pixel’s color, coverage, and Z are stored in the frame buffer. The Video Interface (VI) reads the pixel color and coverage and antialiases the silhouettes of objects.

I would guess that having the gameshark code plus this tool would be the same as disabling anti-aliasing entirely. Which only gives you an extra 1-2 FPS over the reduced version. It wouldn't magically give you "ultra-high" FPS, unfortunately.

I wouldn't call myself an RDP expert, so I could be wrong. If I am, please leave a reply and I'll make a correction to this post.

[Yoshi Milkman]

Hi, I have used your tool and have gotten an error where all the HUD Elements are broken:
https://cdn.discordapp.com/attachments/1...nknown.png
Does it matter if I applied tweaks and ASM patches before using it or anything? I'm using GlideN64 btw.

[CrashOveride]

Hi, I have used your tool and have gotten an error where all the HUD Elements are broken:
https://cdn.discordapp.com/attachments/1...nknown.png
Does it matter if I applied tweaks and ASM patches before using it or anything? I'm using GlideN64 btw.

You don't need this on gliden64/emulation

[Yoshi Milkman]

Hi, I have used your tool and have gotten an error where all the HUD Elements are broken:
https://cdn.discordapp.com/attachments/1...nknown.png
Does it matter if I applied tweaks and ASM patches before using it or anything? I'm using GlideN64 btw.

You don't need this on gliden64/emulation

True, but if I want to have my future hacks be console compatible and have optimum performance, this would be a very useful tool to have. I'd like to find out why it's breaking.

[David]

Hi, I have used your tool and have gotten an error where all the HUD Elements are broken:
https://cdn.discordapp.com/attachments/1...nknown.png
Does it matter if I applied tweaks and ASM patches before using it or anything? I'm using GlideN64 btw.

This tool doesn't edit any game code, so whether you do the tweaks/ASM patches before or after should not matter. 

I can't seem to reproduce the HUD issue with my version of GlideN64. Although it is probably an older version, since the LLE mode has z-buffering issues in vanilla SM64. Can you give me more specifics? Like the specific emulator (including the exact version number), GlideN64 version, and the ASM patches/tweaks you are using? The more detail you can give me, the better.

[Yoshi Milkman]

Hi, I have used your tool and have gotten an error where all the HUD Elements are broken:
https://cdn.discordapp.com/attachments/1...nknown.png
Does it matter if I applied tweaks and ASM patches before using it or anything? I'm using GlideN64 btw.

This tool doesn't edit any game code, so whether you do the tweaks/ASM patches before or after should not matter. 

I can't seem to reproduce the HUD issue with my version of GlideN64. Although it is probably an older version, since the LLE mode has z-buffering issues in vanilla SM64. Can you give me more specifics? Like the specific emulator (including the exact version number), GlideN64 version, and the ASM patches/tweaks you are using? The more detail you can give me, the better.

Well I've tried it on a vanilla and extended ROM and still have the issue. If I try to use Jabo it give me an error code every screen that says "unable to detect microcode settings". I am using this version of Project 64: https://cdn.discordapp.com/attachments/1...64_1.6.zip
And I'm using the version of GlideN64 that came with it. Sorry for the late response, and sorry if this was unhelpful.

[David]

Well I've tried it on a vanilla and extended ROM and still have the issue. If I try to use Jabo it give me an error code every screen that says "unable to detect microcode settings". I am using this version of Project 64: https://cdn.discordapp.com/attachments/1...64_1.6.zip
And I'm using the version of GlideN64 that came with it. Sorry for the late response, and sorry if this was unhelpful.

Ah, ok. Seems like there is a bug with the GlideN64 v4.0 plugin while in HLE mode. If you were using the latest version of Project 64 v2.X, then you could just simply tell people to use LLE mode. You are stuck with using HLE mode with Project64 v1.6. GlideN64 v3.0 does seem to work normally, which you can download here: https://github.com/gonetz/GLideN64/relea...elease_3_0.

Show ContentGlideN64 v3.0 screenshot

But honestly, you should probably not worry about it. This tool is mainly meant to help older ROM hacks get a little better performance on console for little effort. I don't expect many HLE plugins to properly support modified graphics microcodes.

[Yoshi Milkman]

Well I've tried it on a vanilla and extended ROM and still have the issue. If I try to use Jabo it give me an error code every screen that says "unable to detect microcode settings". I am using this version of Project 64: https://cdn.discordapp.com/attachments/1...64_1.6.zip
And I'm using the version of GlideN64 that came with it. Sorry for the late response, and sorry if this was unhelpful.

Ah, ok. Seems like there is a bug with the GlideN64 v4.0 plugin while in HLE mode. If you were using the latest version of Project 64 v2.X, then you could just simply tell people to use LLE mode. You are stuck with using HLE mode with Project64 v1.6. GlideN64 v3.0 does seem to work normally, which you can download here: https://github.com/gonetz/GLideN64/relea...elease_3_0.

Show ContentGlideN64 v3.0 screenshot

But honestly, you should probably not worry about it. This tool is mainly meant to help older ROM hacks get a little better performance on console for little effort. I don't expect many HLE plugins to properly support modified graphics microcodes.

Ah ok. Thank you for your time. I'm actually using a modified microcode for F3DEX2 for better performance, so I most likely won't need this now. But again, thank you so much!

[CrashOveride]

> I'm actually using a modified microcode for F3DEX2 for better performance, so I most likely won't need this now. 
Wait what? a edit of F3DEX2 or just F3DEX2?

Also I highly suggest getting a newer PJ64, preferably a nightly. They are better and more accurate

[PyroJay4]

Side note: i think i explained this to yoshi seperately from this place but i shambled together the files with that version of pj64 he linked
its very inaccurate, but its the most compatible with old af romhacks (sm64e lul) while still looking good, uses some weird version of gliden made by someone named LINK. no documentation or whatever on that gliden version, it just exists in this weird void where it works i guess. honestly i cant even find where i downloaded it anymore. dont worry about whatever really happened with yoshi i guess, its still a really good preconfigured pj64 for compatibility, just not anything worth existing in a dimension that it can be investigated in