Have you tried utilizing either Mupen64plus-gles2n64 or Mupen64plus-gles2rice plugins? I had to mix it up on mine. Some games will work fine on one yet not so much on the other. Takes some time tinkering with each game to find the best fit. Some games just won’t run well at all no matter (even with overclocking).
Reply To: Having issues with N64 ROMs