I’m sure it matters. But how much does it matter? How much (perceived) intelligence would you be willing to sacrifice for an accurate AI predictor?
I’d sacrifice a few %, easily. Maybe 10%. The models are getting smarter at such a fast rate that I’d be willing to lose a month or two of progress to help slow down the AI cheating epidemic.
It sounds like you expect this fingerprinting approach would dramatically reduce the intelligence of their models. But I’m sure anthropic has measured it. I doubt they would have rolled this out if the intelligence cost were that significant. I suspect the cost is less than 5%. I personally can’t tell any difference from before they added fingerprinting. I bet you can’t either.
I’d sacrifice a few %, easily. Maybe 10%. The models are getting smarter at such a fast rate that I’d be willing to lose a month or two of progress to help slow down the AI cheating epidemic.
It sounds like you expect this fingerprinting approach would dramatically reduce the intelligence of their models. But I’m sure anthropic has measured it. I doubt they would have rolled this out if the intelligence cost were that significant. I suspect the cost is less than 5%. I personally can’t tell any difference from before they added fingerprinting. I bet you can’t either.