This is tough. Pretty clearly what's happening here is shot spotter is producing a probability distribution about the places where a shot may have been fired, but they're not showing that information, they're showing a point. That's broken.
It's a shame because setting up some microphones around the city to detect a firearm discharge seems like a great idea. The problem is people are overselling what it is doing.
Something that jumps out at me in the article is this:
Paul Greene, ShotSpotter’s expert witness and an employee of the
company, testified at Simmons’ trial that “subsequently he was asked by
the Rochester Police Department to essentially search and see if there
were more shots fired than ShotSpotter picked up,” according to a civil
lawsuit Simmons has filed against the city and the company. Greene found
a fifth shot, despite there being no physical evidence at the scene that
Simmons had fired. ...
Like, what shotspotter is doing is attempting to resolve the origin of a sound wave in a city. The shockwave from a gun blast must reverberate across buildings all over the place, more than likely you're getting reflected waves rather than waves from the source. You're going to get echos and reverb and it's going to be messy and you're going to be imprecise.
Would shotspotter's AI misclassify noises? Yah. Of course. If it's like any other classifier it's likely listing out a list of probabilities of what something is and, for the sake of simplicity, just saying it's the highest probability output. It's just a filtering step. If you know a gunshot occurred, you update your priors.
Should shotspotter be able to detect the number of gunshots exactly? Maybe? How to differentiate between echos and original? I'm not shocked it can detect the wrong number of shots.
Since it misclassifies noises, isn't exactly sure of the origin, and can read echos as additional shots, is it useless? No.
It's a shame because setting up some microphones around the city to detect a firearm discharge seems like a great idea. The problem is people are overselling what it is doing.
Something that jumps out at me in the article is this:
Like, what shotspotter is doing is attempting to resolve the origin of a sound wave in a city. The shockwave from a gun blast must reverberate across buildings all over the place, more than likely you're getting reflected waves rather than waves from the source. You're going to get echos and reverb and it's going to be messy and you're going to be imprecise.Would shotspotter's AI misclassify noises? Yah. Of course. If it's like any other classifier it's likely listing out a list of probabilities of what something is and, for the sake of simplicity, just saying it's the highest probability output. It's just a filtering step. If you know a gunshot occurred, you update your priors.
Should shotspotter be able to detect the number of gunshots exactly? Maybe? How to differentiate between echos and original? I'm not shocked it can detect the wrong number of shots.
Since it misclassifies noises, isn't exactly sure of the origin, and can read echos as additional shots, is it useless? No.