I’ve found, unfortunately, that even with a proper config file, I have to run retroarch-joyconfig before it’ll respond to my controllers. (Although I may be doing something wrong…I don’t know.)
We may be able to talk you through the button mapping, though. I was confused at first when I saw things like “A (right),” but I figured out that the directions are referring to their standard layout on a SNES or PS controller. The actual directional pad uses axes instead of buttons.
Where are you opening the retroarch.cfg file? I could never edit it on the raspberry pi, but when I plugged the SD card into my laptop I could fiddle with the files using a text editor.