Yes it is possible to use a PS3 pad and an Xbox pad at the same time. Info here of setting up a PS3 pad:
https://github.com/retropie/RetroPie-Setup/wiki/Setting-up-a-PS3-controller
I think if you plug your bluetooth module in the second (lower) usb port it will be assigned /dev/input/js1 instead of js0

So in your retroarch.cfg you would configure player 2 controls like this:

input_player2_joypad_index = “1″
input_player2_a_btn = “13″
input_player2_b_btn = “14″

etc

Here is the list of known working bluetooth adapters: http://elinux.org/RPi_USB_Bluetooth_adapters#Working_Bluetooth_adapters

I don’t think you will be able to hot swap the bluetooth/Xbox pad, either leave both plugged in or swap and reboot.