> It's not possible to engineer an autonomous system that never fails, but it is possible to engineer one in such a way that it never fails to detect that it has failed.
That sounds highly dubious. Here's a hypothetical scenario:
there's a very drunk person on the sidewalk. As a human driver, you know he might act unexpectedly so you slow down and steer to the left. This will help you avoid a deadly collision as the person stumbles into the road.
Now let's take a self driving car in the same scenario, where, since it doesnt have general intelligence, it fails to distinguish the drunk person from a normal pederstrian and keeps going at the same speed and distance from the sidewalk as normally. How, in this scenario, does the vehicle 100% know that it has failed (like you say is always possible)?
That sounds highly dubious. Here's a hypothetical scenario: there's a very drunk person on the sidewalk. As a human driver, you know he might act unexpectedly so you slow down and steer to the left. This will help you avoid a deadly collision as the person stumbles into the road.
Now let's take a self driving car in the same scenario, where, since it doesnt have general intelligence, it fails to distinguish the drunk person from a normal pederstrian and keeps going at the same speed and distance from the sidewalk as normally. How, in this scenario, does the vehicle 100% know that it has failed (like you say is always possible)?