I don’t know how related this is as I don’t own an Xbox controller but I’ve found that player order for my controllers is determined where they are plugged in the USB as follows:
Reply To: Joystick input number changing with different number of controllers