sselphParticipant07/13/2014 at 03:47Post count: 170
I was reinstalling retropie and had to re-scrape my rom directory. For fun, I pulled the full game list for NES and SNES from thegamedb, manually matched everything, and mapped all the hashes to the ID in the DB. Then wrote a quick program to scrape using this data.
I figured I’d share it in case anyone might find it useful:
https://github.com/sselph/scrapersselphParticipant07/13/2014 at 23:15Post count: 170
Odd pretty sure I added a link it’s on github.com/sselph/scraperHarisGuest07/13/2014 at 23:44Post count: 908
Sselph can you write similar script for the other emulators too?sselphParticipant07/14/2014 at 01:53Post count: 170
It is possible to extend the script. I’d have to write code to extract the rom information if the format has headers and other information then use a list of hashes to manually build the mapping from hash to ID. If there is a platform you would like me to try I could take a look.HarisGuest07/14/2014 at 07:15Post count: 908
Sure man. I would like to see it in:
Pretty much for all i suposse:)HarisGuest07/15/2014 at 11:25Post count: 908
I get an
-bash: $: command not found
when i try to run $ go get github.com/sselph/scraper
why is that?sselphParticipant07/15/2014 at 18:40Post count: 170
I wrote the script in go so you’ll have to install that to compile it. Many distributions have packages so should be as easy as something like:
sudo apt-get install golang
You’ll probably need to set a GOPATH environment variable:
But to make things easy I compiled versions for most platforms at:
I recommend running this on a machine other than the pi but I compiled one for it anyway.
(-thumb_only flag can speed it up for testing since it is not having to download/process large images.)deaftolightParticipant07/16/2014 at 01:04Post count: 42
I just get a bunch of errors:
SIGILL: illegal instruction PC=0x3b484 math.init·1() /usr/lib/go/src/pkg/math/pow10.go:34 +0x28 math.init() /usr/lib/go/src/pkg/math/unsafe.go:21 +0x70 fmt.init() /usr/lib/go/src/pkg/fmt/scan.go:1164 +0x70 github.com/sselph/scraper/rom.init() /home/pi/go/src/github.com/sselph/scraper/rom/rom.go:83 +0x7c github.com/sselph/scraper/rom/snes.init() /home/pi/go/src/github.com/sselph/scraper/rom/snes/snes.go:48 +0x70 main.init() /home/pi/go/src/github.com/sselph/scraper/scraper.go:395 +0x74 goroutine 2 [runnable]: created by runtime.main /golang-1.0.2/src/pkg/runtime/proc.c:221 trap 0x6 error 0x0 oldmask 0x0 r0 0x7332f8 r1 0x0 r2 0x0 r3 0x2 r4 0x0 r5 0x0 r6 0x0 r7 0x0 r8 0x0 r9 0x324fec r10 0x10840000 fp 0x732eb6 ip 0xcafebabe sp 0xb6ebcf24 lr 0x3ba18 pc 0x3b484 cpsr 0x20000010 fault 0x0sselphParticipant07/16/2014 at 01:55Post count: 170
I got similar issues when I tried compiling from the raspberry pi. The build command needed ‘GOARM=5 go build <blah>’ but seems the version of go in raspbian is older(1.0.2 instead of 1.1+ I’m using 1.2) and couple things don’t seem to work.
I pushed a version that fixes one of the issues but the library that resizes images doesn’t work so you’ll have to use -thumb_only so resizing isn’t needed. Use go get -u github.com/sselph/scraper to update the package.
I also cross compiled a version for the raspberry pi using 1.2 here. Or if you like you can do that yourself from an ubuntu machine by installing golang-go-linux-arm and doing ‘GOARM=6 GOOS=linux GOARCH=arm go build github.com/sselph/scraper’
https://github.com/sselph/scraper/releases/download/v0.1-beta/scraper_rpi.zipgrassmunkParticipant11/29/2014 at 10:52Post count: 5
I just wanted to say thanks for this! You’ve saved me a tonne of time on scraping material!FloobMember01/12/2015 at 22:54Post count: 1629
Is there anyway it could deal with Atari Lynx games in the future?sselphParticipant01/12/2015 at 23:10Post count: 170
I actually added that over the weekend try v0.5.8-beta and let me know if it is working correctly.
- The forum ‘Everything else related to the RetroPie Project’ is closed to new topics and replies.