I had the same issue using “zero delay encoders”. The problem is that they aren’t “true” keyboard encoders, but are seen as generic “joysticks”. I used adafruit’s guide and used their retrogame tutorial and program to program a GPIO button (much like Matthew did) to key “ESC”.
Here is the link: https://learn.adafruit.com/retro-gaming-with-raspberry-pi/buttons
Works like a champ and gets me out of FBA/Mame every time.
read the instructions CAREFULLY.