Viewing 14 posts - 1 through 14 (of 14 total)
  • Author
    Posts
  • sselph
    Participant
    Post count: 170

    Hi,

    I was reinstalling retropie and had to re-scrape my rom directory. For fun, I pulled the full game list for NES and SNES from thegamedb, manually matched everything, and mapped all the hashes to the ID in the DB. Then wrote a quick program to scrape using this data.

    I figured I’d share it in case anyone might find it useful:
    https://github.com/sselph/scraper

    deaftolight
    Participant
    Post count: 42

    where is it

    sselph
    Participant
    Post count: 170

    Odd pretty sure I added a link it’s on github.com/sselph/scraper

    Haris
    Guest
    Post count: 908

    Sselph can you write similar script for the other emulators too?

    sselph
    Participant
    Post count: 170

    It is possible to extend the script. I’d have to write code to extract the rom information if the format has headers and other information then use a list of hashes to manually build the mapping from hash to ID. If there is a platform you would like me to try I could take a look.

    Haris
    Guest
    Post count: 908

    Sure man. I would like to see it in:

    Fba,turbograf16,mame,genesis,megadrive

    atari2600,gbc

    Pretty much for all i suposse:)

    Haris
    Guest
    Post count: 908

    I get an

    -bash: $: command not found

    when i try to run $ go get github.com/sselph/scraper

    why is that?

    sselph
    Participant
    Post count: 170

    I wrote the script in go so you’ll have to install that to compile it. Many distributions have packages so should be as easy as something like:
    sudo apt-get install golang
    You’ll probably need to set a GOPATH environment variable:
    http://golang.org/doc/code.html#GOPATH

    But to make things easy I compiled versions for most platforms at:
    https://github.com/sselph/scraper/releases

    I recommend running this on a machine other than the pi but I compiled one for it anyway.

    (-thumb_only flag can speed it up for testing since it is not having to download/process large images.)

    deaftolight
    Participant
    Post count: 42

    I just get a bunch of errors:

    SIGILL: illegal instruction
    PC=0x3b484
    
    math.init·1()
            /usr/lib/go/src/pkg/math/pow10.go:34 +0x28
    math.init()
            /usr/lib/go/src/pkg/math/unsafe.go:21 +0x70
    fmt.init()
            /usr/lib/go/src/pkg/fmt/scan.go:1164 +0x70
    github.com/sselph/scraper/rom.init()
            /home/pi/go/src/github.com/sselph/scraper/rom/rom.go:83 +0x7c
    github.com/sselph/scraper/rom/snes.init()
            /home/pi/go/src/github.com/sselph/scraper/rom/snes/snes.go:48 +0x70
    main.init()
            /home/pi/go/src/github.com/sselph/scraper/scraper.go:395 +0x74
    
    goroutine 2 [runnable]:
    created by runtime.main
            /golang-1.0.2/src/pkg/runtime/proc.c:221
    
    trap    0x6
    error   0x0
    oldmask 0x0
    r0      0x7332f8
    r1      0x0
    r2      0x0
    r3      0x2
    r4      0x0
    r5      0x0
    r6      0x0
    r7      0x0
    r8      0x0
    r9      0x324fec
    r10     0x10840000
    fp      0x732eb6
    ip      0xcafebabe
    sp      0xb6ebcf24
    lr      0x3ba18
    pc      0x3b484
    cpsr    0x20000010
    fault   0x0
    
    sselph
    Participant
    Post count: 170

    I got similar issues when I tried compiling from the raspberry pi. The build command needed ‘GOARM=5 go build <blah>’ but seems the version of go in raspbian is older(1.0.2 instead of 1.1+ I’m using 1.2) and couple things don’t seem to work.

    I pushed a version that fixes one of the issues but the library that resizes images doesn’t work so you’ll have to use -thumb_only so resizing isn’t needed. Use go get -u github.com/sselph/scraper to update the package.

    I also cross compiled a version for the raspberry pi using 1.2 here. Or if you like you can do that yourself from an ubuntu machine by installing golang-go-linux-arm and doing ‘GOARM=6 GOOS=linux GOARCH=arm go build github.com/sselph/scraper’
    https://github.com/sselph/scraper/releases/download/v0.1-beta/scraper_rpi.zip

    grassmunk
    Participant
    Post count: 5

    I just wanted to say thanks for this! You’ve saved me a tonne of time on scraping material!

    Floob
    Member
    Post count: 1629

    Is there anyway it could deal with Atari Lynx games in the future?

    sselph
    Participant
    Post count: 170

    I actually added that over the weekend try v0.5.8-beta and let me know if it is working correctly.

    Floob
    Member
    Post count: 1629

    Great! I’ll give it a go. Thanks :)

Viewing 14 posts - 1 through 14 (of 14 total)
  • The forum ‘Everything else related to the RetroPie Project’ is closed to new topics and replies.