There’s a special tool you can run in retropie-setup/config for setting up the PS3 controllers and Dongle. Have you used this?
I have done this before. To sync the controllers with the Dongle, I had to attach the controllers via USB. You’ll need a USB hub to do this as you need to have the keyboard, dongle, and controller attached all at once…
