Not quite what I’d expect, but try updating your controller file to read like this.
(Do you have a link – amazon maybe? – to your controller?)

input_device = "DragonRise Inc. Generic USB Joystick "
input_driver = "udev"
input_start_btn = "8"
input_exit_emulator_btn = "8"
input_up_axis = "-1"
input_a_btn = "0"
input_b_btn = "1"
input_reset_btn = "1"
input_down_axis = "+1"
input_right_axis = "+0"
input_state_slot_increase_axis = "+0"
input_x_btn = "5"
input_menu_toggle_btn = "5"
input_select_btn = "4"
input_enable_hotkey_btn = "4"
input_y_btn = "2"
input_left_axis = "-0"
input_state_slot_decrease_axis = "-0"

Then to exit a game, hold your select button and tap Start with your P1 controller.