I could weigh in on a few things here.

Firstly, think of the /all/ config as the “master” config. If you look at the configs in the emulator specific folders, you will notice they are mostly empty. The way this works, is that it looks at the emulator specific config first for settings, and whatever it doesn’t find, it will take from the global config. So, if you want to use one global config for everything, just set up the one in /all/ and ignore the rest.

As for your issue, I’d say its likely you still have the keyboard keys active in the cfg as well as your joypad ones. make sure to go through all the keys under the keyboard section and put a # infront of every single one of them to disable it. I have recently uploaded my retroarch.cfg for use in a different thread so you can look at mine and compare.