Now that picodrive has replaced dgen as the default megadrive emulator, I don’t see any reason to use dgen. I was never able to get my controllers working with dgen, but picodrive uses your retroarch settings. then, if you want to do a custom mapping for megadrive (to use xyz) you just edit the retroarch.cfg file located in /home/pi/RetroPie/Configs/MegaDrive

you may already know all this but it might be helpful for others:
Look at the retroarch.cfg file in the Configs/All directory and copy your settings to the clipboard. then edit the retroarch.cfg file in Configs/MegaDrive (which is blank). copy your controller settings into it. Now you can edit your button assignments. Anything in this retroarch.cfg file that’s different from the one in configs/all will override the settings for that particular emulator.