ES Scraper Setup

[Arnaud]

Hello,

I have tons of roms and running the scraper from the emulation station seems to make the system crash (memory problems ?). I am forced to rerun the process many times and it takes forever. I’ve heard there is a way to run this from the command line ? Do you think that may help ? Or am I facing a memory problem if I do so also ?

I am running the latest distro of Retro Pie but cannot find the ES Scraper from the Script Menu. Is there something to do to make it available ?

Thanks a lot !

[JayBi]

Same problem here. The scraper in the ES is working for a while, but more than 500 games and the ES GUI became extremely laggy and it freeze from time to time.
I installed the PIL stuff (for resizing problem) and also searched for the solution described here
but i didn’t manage to find the scraper.py. It’s not in /home/pi/RetroPie/supplementary/ES-scraper/scraper.py anymore.
Any idea?

[sfalexi]

If you’re using 2.x, a lot of the folders have moved from the HOME folder to the root folder and then /opt

This is what I’m doing.

Until I’m fully setup, I put the config file on the SD Boot in hdmi_safe=1

This puts it (I believe) in 480i. I figure this should tax raspberry pi the least and give it its best shot to run for a while without crashing.

I run the scraper one system at a time. So for SNES, I run it. I disable the user verification. And I have it plugged into an ethernet cable.

I have the same problem, about 150 – 250 games it’ll crash or give an error about not enough memory. But you know what? Run it overnight. Wake up, restart it, and it’ll pick up where it left off and run it as you go off to work. Come back home, crashed again, but now it’s 400 games in. Run it again. It’ll crash or pause again, but as you go to bed and run it again, it’ll finish. 1000ish games in a day and a half.

There will be missing games and errors cause it assumes the first hit is the right now. Go to the home folder and there should be a .emulationstation folder or something like that. There’ll be a gameslist folder, then my SNES folder, and then a gameslist.xml. Browse it and delete any errors. I just checked the game file name and checked what the new emulationstation title is. If they don’t match, delete all that data starting with <game> and ending with </game>

At that point, I rerun the scraper, but make sure I try to verify each game. 70 games left, after manually scraping I can cut that down to about 35. And now the fun begins where I try to change the filenames to something that can be accurately scraped.

Two days and I’m down to 35 games? Heck! I’m happy!

[Jaybi]

Thanks,
I’ve checked for scraper.py in all the possible directories also using
sudo find * | grep scraper
but i didn’t find it.
Even when i manage to scrape (within the ES interface) more than 500 games, the system become laggy, not responsive and it crashes from time to time. This is, I guess, due to the size of the pictures it has to deal with.
I would try to solve it resizing the pictures, but so far, i didn’t manage to find the proper way to resise them.

[Arnaud]

I think it can be downloaded

[boelze]

I have had the same problem. Here is what I did to solve it:
I installed the old Version of RetroPie 1.1 where the scaper can still be accessed through the setup script, copied over my roms and ran the scraper (took around 5 hours for over 1,5k roms). Then I backupped the pictures and .xml files on my laptop, installed the new RetroPie image and copied them over again.
Works like a charm. I don’t understand why the scraper was removed from the setup script though…

If anyone can explain to me how to start the scraping script from console on the new RetroPie image I would highly appreciate it!

[Floob]

Looks like command line can be run from this approach?

Section 5.3
https://github.com/retropie/RetroPie-Setup/wiki/RetroPie-Arch-Linux-Flavor

Section gamelist.xml
https://github.com/Aloshi/EmulationStation

[Author]

Posts