What type of controller is it? How is it connected (via usb/ via bluetooth/ wireless receiver)? YOu say re-plug which suggests its just a wired USB but then you say ‘doesn’t get recognized properly’ which suggests the controller itself is giving some kind of power indication?
If its occasionally is it always in the same USB port it fails? Does it fail on all of them?
have you also checked the JS inputs? Perhaps if it is recognized but you cannot use it its because its not on the index you are expecting.