I got 93% agreement based on a simple rule: It's a vehicle if it is touching the ground and powered with a motor.
And if I were moderating a community, which I have done a lot of in the past, I would clarify the rules as soon as this disagreement popped up. This isn't a great experiment because it doesn't involve any judgement calls -- you can apply a (pretty simple) objective ruleset here.
The main areas where I disagreed with the majority are a police car and ambulance. The police car and ambulance are clearly vehicles. People who said no were applying other rules. But again that rule is simple -- emergency services are exempt when responding to emergencies.
Moderating actual communities is a lot more difficult because there aren't a simple objective set of rules you can apply. Judgement calls are necessary. We ran this experiment on reddit for a while. We would present the user with a link and ask them "is this spam". It was rare for a link to get 100% (or even 98%) yes. Only the very most obvious cases. Otherwise most links would be closer to 60-70% if it were "spammy". And only things like wikipedia would get close to 0%. Everything else in between was a spectrum.
That is what makes moderating hard.
The conclusion of this experiment is correct, but I would say the methods are deeply flawed.
And if I were moderating a community, which I have done a lot of in the past, I would clarify the rules as soon as this disagreement popped up. This isn't a great experiment because it doesn't involve any judgement calls -- you can apply a (pretty simple) objective ruleset here.
The main areas where I disagreed with the majority are a police car and ambulance. The police car and ambulance are clearly vehicles. People who said no were applying other rules. But again that rule is simple -- emergency services are exempt when responding to emergencies.
Moderating actual communities is a lot more difficult because there aren't a simple objective set of rules you can apply. Judgement calls are necessary. We ran this experiment on reddit for a while. We would present the user with a link and ask them "is this spam". It was rare for a link to get 100% (or even 98%) yes. Only the very most obvious cases. Otherwise most links would be closer to 60-70% if it were "spammy". And only things like wikipedia would get close to 0%. Everything else in between was a spectrum.
That is what makes moderating hard.
The conclusion of this experiment is correct, but I would say the methods are deeply flawed.