I don't think that would work due to "bad" drivers. We all drive differently than we know we should drive in certain circumstances (e.g. road is completely empty in the middle of rural new mexico)
For example, you can imagine FSD would determine to go straight down a straight lane with no obstacles - that would be the correct behavior. Now imagine in real life the driver takes their hand off the wheel to adjust the radio or AC, and as a result the car drifts over and lightly cross the lane marker - this doesn't really matter because it's broad daylight and the driver can see there's nothing but sand and rocks for 2 miles all around them. What's the machine conclude?
For example, you can imagine FSD would determine to go straight down a straight lane with no obstacles - that would be the correct behavior. Now imagine in real life the driver takes their hand off the wheel to adjust the radio or AC, and as a result the car drifts over and lightly cross the lane marker - this doesn't really matter because it's broad daylight and the driver can see there's nothing but sand and rocks for 2 miles all around them. What's the machine conclude?