rapidata

it is surprising, but give this question to some random people on the street without context and you would be surprised

All good ^^, its a fair point, we have come up with some fun ways to track peoples reliability over time. But the validation sets contain plenty of forced-choice questions, those that have an empirical true can be used directly to calculate a reliability, those that are subjective need to be re-asked after sometime to ensure consistency. People that don't pass thresholds would not be part of the 10'000 here.

But of course. If every human was told to take 3 minutes to deeply think about it and told that its a trick question, then they most likely will all get it right. But its the same with the LLMs, if you ask them like that they will get it right most of the time. The low effort is kinda the point here.

All sorts, we go through third parties. But apps include stuff like Duo Lingo, Games, Sport Betting Apps ect. Its an optional opt in instead of watching ads or paying for the app. And obviously you are vetted that you don't spam.

We were surprise ourselfes, but if you walk around and randomly ask people in the street, I think you would be surprised what you would find. Its a trick question.

We try a bit harder than that my friend.