Despite going behind to a Darren Mills goal after 13 minutes, the hosts hit back with two strikes from Ryan Crisp on 26 and 56 minutes to take a 2-1 lead at Aldiss Park.
After Tarryn Allarakhia grabbed an equaliser for the visitors just three minutes later on the hour mark, Lewis Butcher won it for the Magpies with a minute to go.
The win lifted Dereham to eighth spot and to within four points of the play-off spots.
Follow us: @rymanleaguenews on Twitter | Ryman League group on Facebook
Keep up-to-date with our exclusive email newsletters.
Subscribe