Same here, but I think the xin-most joystick is sending the same “turbo” joystick movements in mame as well, because when playing a test game (donkey kong) Mario moves really slow and jerky, as if the right and left movements are being repeated over and over.
The weird part for me is that after coming into retropie from a fresh flash of the sd card, everything seems fine, but when I come back to it after a couple hours the UI movement is uncontrollable as you described, sending multiple up/down/left/right actions for every joystick movement.
Ryan