Page 1 of 1

7zip file compression support

Posted: Wed Aug 13, 2008 4:49 pm
by erroneus
I'm something of a newb in that I have only recently been turned on to the Snes9x project and I must say I am very impressed in general. I think clearer documentation (for Linux at least) is probably needed since I *still* cannot figure out exactly what is needed to properly set up my config file to work with my USB-to-SNES controller adapter button mapping. But with that said, after a couple of days frustration, I learned of Snes9x-gtk! I haven't looked back since. (I would love it if Fedora RPMS in 64 bit were available from a yum repository, but just having a 32bit compiled binary is good for now.)

Now on to the real matter.

I have been acquiring SNES ROM image files from various sources and I found one that claims to be "every ROM ever in existence." The trouble is that these ROMs are compressed in 7zip format. So I wrote a quick shell script to convert the 7z files over to zip. It works perfectly. Then I saw the truly bad news...

...7zip files are anywhere from 4 to 7 times smaller than zip files! A 7z file of less than a megabyte converted to zip often results in a 4MB zip file. That's a tremendous difference.

So while it's trivial to convert the compressed files from one format to another, the difference in size is NOT trivial. I think any plans to add 7zip format to the project should be accelerated as the benefit should be obvious.

As to snes9x-gtk? This should be rolled into the project! It's awesome. It simply works the way any good program should. It doesn't trouble the user with config files or anything so cumbersome.

Posted: Wed Aug 13, 2008 5:18 pm
by zones
Use NSRT and the JMA format.

Posted: Thu Aug 14, 2008 9:45 pm
by erroneus
Okay cool. I guess it would mean weeding through every last 7z file to pull out the redundancies (there are over 1800 of these 7z files) but perhaps I could modify my script to use either the newest file or the oldest file when converting.

Question: Where can I find this JMA compression tool? I've googled for it and only found one reference... not a good one. I'll try digging around on the forum here too, but initial searches yielded listings with some VERY long multipaged threads so it looks like it may take a while to find more information on this NSRT and JMA stuff.

====

Okay, just found some information on NSRT. You realize that's for Windows only right? Is there a Linux version of this?

In any case, you're probably right about the way the 7zip archive file is packed and that only minimal compression loss would be experienced if I were to rewrite the script to only compress one of the extracted files... either the newest or the oldest by file date. So for now, perhaps that's exactly what I should do.

Posted: Fri Aug 15, 2008 12:25 am
by Deathlike2
There is a Unix/Linux port... and a Mac port to boot. Please look harder next time. http://snesemu.black-ship.net/index.php?page=tools

Posted: Fri Aug 15, 2008 6:52 am
by squall_leonhart69r
byuu wrote:The only reason you'd see a size difference of 400% is because you're getting ROMs from the idiots over at edgeemu or somesuch that insist on sticking every last hacked-up copy of every game in existence into solid archives. Eg if you open the 7z file, you'll see multiple variations of the same game, where most of them are nothing more than minor sprite hacks and header byte changes.

Packing all of those images into one archive is asking for trouble anyway, as Snes9X has no way of letting you pick which one you want to actually load. And I really can't imagine why anyone would want the 13 header variations of Super Mario World that Cowering indexes.

Even in the most extreme cases, 7z should not be more than ~30-40% more effective than ZIP for single ROM, and if you have examples where it reaches >100%, please do share the image names and sizes you get, as I would love to compare for myself.

And lastly, if you are still worried about space, JMA is roughly as good as 7z. Certainly better than ZIP, and it's supported by most emulators.
comparing Zip to 7zip, when using ultra lmza compression, 7zip wins hands down, however since the size difference is in KB and not MB, does it really matter?

Posted: Wed Feb 02, 2011 11:23 am
by andwan0
I use snes9X for the PSP and due to limited memory card sizes, 7-Zip would seem the most ideal way to save space.

Please please can Snes9X get 7-Zip support.

Posted: Wed Feb 02, 2011 3:43 pm
by Camo_Yoshi
squall_leonhart69r wrote:
byuu wrote:The only reason you'd see a size difference of 400% is because you're getting ROMs from the idiots over at edgeemu or somesuch that insist on sticking every last hacked-up copy of every game in existence into solid archives. Eg if you open the 7z file, you'll see multiple variations of the same game, where most of them are nothing more than minor sprite hacks and header byte changes.

Packing all of those images into one archive is asking for trouble anyway, as Snes9X has no way of letting you pick which one you want to actually load. And I really can't imagine why anyone would want the 13 header variations of Super Mario World that Cowering indexes.

Even in the most extreme cases, 7z should not be more than ~30-40% more effective than ZIP for single ROM, and if you have examples where it reaches >100%, please do share the image names and sizes you get, as I would love to compare for myself.

And lastly, if you are still worried about space, JMA is roughly as good as 7z. Certainly better than ZIP, and it's supported by most emulators.
comparing Zip to 7zip, when using ultra lmza compression, 7zip wins hands down, however since the size difference is in KB and not MB, does it really matter?
Well, in the even that you have hundreds and hundreds of roms, it can all add up.

I second the idea of having built-in 7zip support for Snes9x; It probably wouldn't be too hard because we could just integrate the un7z.dll.

Forgive me if I'm not thinking the right way... I just started learning how to use VisualC++. :P

Posted: Mon Feb 07, 2011 2:53 pm
by andwan0
Okay, I know you can't integrate 7-Zip, but what about bundling 7-Zip command-line (7z.exe & 7z.dll) with SNES9X then use it as an axillary step to decompress 7-Zip files before game load... then remove the files afterwards?

Posted: Tue Feb 15, 2011 6:19 pm
by SparroHawc
7Zip is under LGPL, actually. You -could- legitimately include it, as long as you don't integrate its source into SNES9x proper and only access it as an external library. You'd have to include a notice in the About screen, and specify in any source that calls the library that it does so; that's about it. Well, that and including a copy of the GPL and LGPL (preferrably while specifying that it applies only to 7Zip).

--edit--
as long as you don't use 7Z.dll - that includes some extra stuff from unRAR.