tolugenius

Curious on two things

1. How often do you encounter false positives, and does that change things?

2. Have you considered outright prohibiting/banning vs disclosure?

Just want to hear from someone running a platform how they deal with this since I imagine it's very hard to get right.

There is probably far, far more people trusting benchmarks and "I remade x thing in 1 prompt with y model" claims than you'd imagine, just ignore all of it. You know your workflow and what better should be and could be, measure on what works for you. You should note (and I may be wrong, not active in these part) a lot of those demos are very toy, recreating a known game, known app, known workflow, etc. very interesting but again a very toy example that should be taken with that in mind.

thanks!

Do you mind sharing them and what they do?

I wonder what would "compute" mean if cpus were more efficient at matrix multiplication say 15 years ago. And on the flipside, what it would take to say train a frontier model entirely on cpus in the future.