I wouldn't build it up with glue and filler. That's going to be messy. Either rout it all down to the lower level and then use door seal to bring level up (I'm assuming this is a removable front baffle). Alternatively glue in a whole straight strip of wood of appropriate thickness. Or start...