7zip file compression support

This is for people involved in the developement of Snes9x, or SNES emulators in general.
BUG REPORTS BELONG IN TECH SUPPORT/BUG TRACKING!
Post Reply
erroneus
Snes9x White Belt
Posts: 2
Joined: Wed Aug 13, 2008 3:41 pm
Location: Arlington, TX, USA

7zip file compression support

Post by erroneus »

I'm something of a newb in that I have only recently been turned on to the Snes9x project and I must say I am very impressed in general. I think clearer documentation (for Linux at least) is probably needed since I *still* cannot figure out exactly what is needed to properly set up my config file to work with my USB-to-SNES controller adapter button mapping. But with that said, after a couple of days frustration, I learned of Snes9x-gtk! I haven't looked back since. (I would love it if Fedora RPMS in 64 bit were available from a yum repository, but just having a 32bit compiled binary is good for now.)

Now on to the real matter.

I have been acquiring SNES ROM image files from various sources and I found one that claims to be "every ROM ever in existence." The trouble is that these ROMs are compressed in 7zip format. So I wrote a quick shell script to convert the 7z files over to zip. It works perfectly. Then I saw the truly bad news...

...7zip files are anywhere from 4 to 7 times smaller than zip files! A 7z file of less than a megabyte converted to zip often results in a 4MB zip file. That's a tremendous difference.

So while it's trivial to convert the compressed files from one format to another, the difference in size is NOT trivial. I think any plans to add 7zip format to the project should be accelerated as the benefit should be obvious.

As to snes9x-gtk? This should be rolled into the project! It's awesome. It simply works the way any good program should. It doesn't trouble the user with config files or anything so cumbersome.
User avatar
zones
Official Mac OSX Porter/Dev
Posts: 908
Joined: Sat May 15, 2004 11:58 pm
Location: Japan

Post by zones »

Use NSRT and the JMA format.
erroneus
Snes9x White Belt
Posts: 2
Joined: Wed Aug 13, 2008 3:41 pm
Location: Arlington, TX, USA

Post by erroneus »

Okay cool. I guess it would mean weeding through every last 7z file to pull out the redundancies (there are over 1800 of these 7z files) but perhaps I could modify my script to use either the newest file or the oldest file when converting.

Question: Where can I find this JMA compression tool? I've googled for it and only found one reference... not a good one. I'll try digging around on the forum here too, but initial searches yielded listings with some VERY long multipaged threads so it looks like it may take a while to find more information on this NSRT and JMA stuff.

====

Okay, just found some information on NSRT. You realize that's for Windows only right? Is there a Linux version of this?

In any case, you're probably right about the way the 7zip archive file is packed and that only minimal compression loss would be experienced if I were to rewrite the script to only compress one of the extracted files... either the newest or the oldest by file date. So for now, perhaps that's exactly what I should do.
Deathlike2
Snes9x Brown Belt
Posts: 1158
Joined: Mon Jan 10, 2005 6:34 am

Post by Deathlike2 »

There is a Unix/Linux port... and a Mac port to boot. Please look harder next time. http://snesemu.black-ship.net/index.php?page=tools
squall_leonhart69r

Post by squall_leonhart69r »

byuu wrote:The only reason you'd see a size difference of 400% is because you're getting ROMs from the idiots over at edgeemu or somesuch that insist on sticking every last hacked-up copy of every game in existence into solid archives. Eg if you open the 7z file, you'll see multiple variations of the same game, where most of them are nothing more than minor sprite hacks and header byte changes.

Packing all of those images into one archive is asking for trouble anyway, as Snes9X has no way of letting you pick which one you want to actually load. And I really can't imagine why anyone would want the 13 header variations of Super Mario World that Cowering indexes.

Even in the most extreme cases, 7z should not be more than ~30-40% more effective than ZIP for single ROM, and if you have examples where it reaches >100%, please do share the image names and sizes you get, as I would love to compare for myself.

And lastly, if you are still worried about space, JMA is roughly as good as 7z. Certainly better than ZIP, and it's supported by most emulators.
comparing Zip to 7zip, when using ultra lmza compression, 7zip wins hands down, however since the size difference is in KB and not MB, does it really matter?
andwan0
Snes9x White Belt
Posts: 7
Joined: Wed Feb 02, 2011 11:20 am

Post by andwan0 »

I use snes9X for the PSP and due to limited memory card sizes, 7-Zip would seem the most ideal way to save space.

Please please can Snes9X get 7-Zip support.
User avatar
Camo_Yoshi
Snes9x Purple belt
Posts: 922
Joined: Thu Nov 08, 2007 7:59 pm

Post by Camo_Yoshi »

squall_leonhart69r wrote:
byuu wrote:The only reason you'd see a size difference of 400% is because you're getting ROMs from the idiots over at edgeemu or somesuch that insist on sticking every last hacked-up copy of every game in existence into solid archives. Eg if you open the 7z file, you'll see multiple variations of the same game, where most of them are nothing more than minor sprite hacks and header byte changes.

Packing all of those images into one archive is asking for trouble anyway, as Snes9X has no way of letting you pick which one you want to actually load. And I really can't imagine why anyone would want the 13 header variations of Super Mario World that Cowering indexes.

Even in the most extreme cases, 7z should not be more than ~30-40% more effective than ZIP for single ROM, and if you have examples where it reaches >100%, please do share the image names and sizes you get, as I would love to compare for myself.

And lastly, if you are still worried about space, JMA is roughly as good as 7z. Certainly better than ZIP, and it's supported by most emulators.
comparing Zip to 7zip, when using ultra lmza compression, 7zip wins hands down, however since the size difference is in KB and not MB, does it really matter?
Well, in the even that you have hundreds and hundreds of roms, it can all add up.

I second the idea of having built-in 7zip support for Snes9x; It probably wouldn't be too hard because we could just integrate the un7z.dll.

Forgive me if I'm not thinking the right way... I just started learning how to use VisualC++. :P
Snes9x FAQs | Forum Rules
What operating system are you using? 32 or 64bit? Version of Snes9x? Is the text at the bottom of the window white when you load the game?
These suggestions are usually the solution to your issue!
andwan0
Snes9x White Belt
Posts: 7
Joined: Wed Feb 02, 2011 11:20 am

Post by andwan0 »

Okay, I know you can't integrate 7-Zip, but what about bundling 7-Zip command-line (7z.exe & 7z.dll) with SNES9X then use it as an axillary step to decompress 7-Zip files before game load... then remove the files afterwards?
SparroHawc
Official Android Porter
Posts: 303
Joined: Mon Feb 07, 2011 9:20 pm
Location: All up in your business

Post by SparroHawc »

7Zip is under LGPL, actually. You -could- legitimately include it, as long as you don't integrate its source into SNES9x proper and only access it as an external library. You'd have to include a notice in the About screen, and specify in any source that calls the library that it does so; that's about it. Well, that and including a copy of the GPL and LGPL (preferrably while specifying that it applies only to 7Zip).

--edit--
as long as you don't use 7Z.dll - that includes some extra stuff from unRAR.
Post Reply