The video above shows you where the controller files are held.

/opt/retropie/emulators/retroarch/configs/

Go there to find the one for your PS3 controller – they are named fairly well. Then change the value for your hotkey enable button.

Just say if anything in the video isn’t clear.

If you want, you can delete the pre-made files and run the retropie setup script to re-register your controller.