Thanks to EBZero for his post at the RetroPie subreddit for pointing me in the right direction (https://www.reddit.com/r/RetroPie/comments/3p5pkq/beware_using_select_as_a_joypad_hotkey_in_nes/).

The problem arises because the Select key is used in Star Tropics to navigate down in the game menus as well as being the hotkey. The easiest way to get around this is to remap the Select key to another button.

1. Launch Star Tropics from ES and before it loads, press any key or js0 to bring up the runcommand menu, then select option 9 (Edit custom RetroArch config for this rom).

2. Add the following line to the custom config file:

input_player1_select_btn = "x"
where x is the number of the button you wish to use for Select.

3. Now exit and launch the rom. You should now be able to enter a name and start a game using the remapped Select button.

This works in lr-fceumm but not in lr-nestopia. I haven’t tested Star Tropics 2 but I assume it should work with the same method.

I remap the hotkey instead of select. I use a PS4 controller and remap the hotkey to 12 which is the guide button.

From what I remember you go into command by F4 then type;

jstest

I think it is that. I’m sure someone will correct me. With that command you can see what number buttons are being pressed.

used 12 for ps3 controller and it its triangle and it works now