I just added the following to gpsp_input.cfg:
btn_quit = 0
I used the X button on my SNES controller. Since the GBA only had two buttons, the X and Y buttons are unused. No need to use your Select button (which is used more often than you might think in GBA games.)
I haven’t yet figured out if there is a hotkey I can add for save or load states, but SRAM saves work fine.