Gooder DATs

I use a program called ClrMAME Pro to audit the game ROMs that I have in my collection against a list of known ROMs. The list of know ROMs is stored in a file called a DAT file, and there are different DAT files for each system for which I have ROMs. For the bulk of the gaming consoles and handhelds that I am interested in emulating, the DATs are produced from a set of tools called "GoodTools" made by some guy called Cowering.

The trouble with GoodTools is that they are intended to be used to identify unknown ROMs, not for auditing a collection. If a ROM were to be dumped poorly such that it wouldn't work in an emulator, the checksum for that fle would still be found in GoodTools so that you could identify it as a bad ROM. People have a tendancy to use GoodTools to monitor the completeness of their ROM collection, and the result is that when you download a set of ROMs for a given system, it tends to have a bunch of broken ROMs included with it which just wastes space. This can be a real problem for systems with lots of known bad dumps and/or large ROM sizes. The quintessential example is the Nintendo 64, which has ROM sizes around ~10MB and lots of known bad dumps/overdumps/hacks/modified ROMs. Here are the known versions of the ROM for the game "1080 Snowboarding" for the N64:

As you can see, there are two valid versions of the game, a version released in Europe that includes multilanguage support for 4 languages and a version released in Japan and the United States. The first of these includes two bad dumps and two "fixed" roms (modified to work with a particular imperfect emulator). For the second version there are 10 bad dumps, 7 fixed roms, and two hacked roms. The grand total, then, is 25 versions of this ROM, only 2 of which are valid. And at ~11MB per version, that's about 253MB wasted on this game. Even worse is the fact that 12 of these versions (~132MB) will never work on any emulator ever, as they are broken dumps. I finally got fed up with having all this crap data on my harddrive and wrote some PERL to cull the chaff from my DAT files.

I eliminated anything that was a broken dump, an overdump, a fixed rom, a translation, a trained rom (modified to allow cheating), a cracked rom, or anything with a bad checksum. While I was at it, I nuked all the public domain ROMs (playable ROMs that were never available as actual games for the system). I really just want to be able to emulate what could have been played on the original hardware.

Implementing the new DATs in ClrMAME has resulted in a massive reduction in junk on my harddrive. The four GoodTools systems with the largest ROM collections are Nintendo 64 (27GB), Super Nintendo (10GB), Gameboy Advance (7GB), and Sega Genesis (3GB). The combined total size taken up by the GoodTools complete sets for these four systems is almost 50 gigabytes. After implementing the new MouserDATs, they drop down in size to 11GB, 3GB, 5GB, and 1GB, respectively, for a grand total of about 20 gigabytes and a reduction of 60% in size.

Yeah so I nuked 30GB of crap from my harddrive without losing any desired functionality from my emulators.

The process of producing the new DAT files and re-scanning the ROMs takes some time, so I haven't done it for all of the GoodTools yet. But the ROMs audit page now reflects the new DATs where I've implemented them.

"Gooder DATs" Comments

Hey nice work! The good thing is, it's unlikely the DAT files will change much in the future. Most additions to them these days are yet another bad dump or some hack or something.

Yeah I agree. Almost all of the old systems, for which games are no longer being made, have no missing ROMs (i.e. all of the games ever made for the system have been dumped successfully). This means that any additions in the future will be public domain or broken or hacks, none of which I want. So once I get a system up to 100% on the MouserDAT, I've basically got everything I'll ever want for that system—leaving me free to complain about the lack of consistency between emulators. :)


Post a comment
Name Required
Email Required

URL
Remember info
Yes No

Comment Required


Type the characters you see in the picture above.