Code 4 almost always means not enough disk space. You can have a large card but not have the space expanded. Retropie 3.5 should expand automatically but previous versions you had to expand your filesystem with the sudo raspi-config menu. Try that and see if it makes a difference.
Possible you also have a bad SD card but the lack of space is more likely.