Not if your system is installed in a car which normally travels on roads...
A simple Particle Filter conditioned on "90% chance the car is on a road or other drivable space" will robustly get rid of any error accumulation, leaving you a position estimate that may even typically be more accurate than GPS.
A simple Particle Filter conditioned on "90% chance the car is on a road or other drivable space" will robustly get rid of any error accumulation, leaving you a position estimate that may even typically be more accurate than GPS.