I've noticed something else - as Anthropic models get even more and more superhuman, they seem to serve me more and more casual nonsense.
Not like adding glue to pizza. Here's an example from today (paraphrasing): "you need to run `git merge-base branch1 branch2`. Pay attention to the order of arguments, it is important: `git merge-base` is symmetric and returns the same value regardless of the order of inputs".
So which one is it? Symmetric or not? It's not even one of those where it self-corrects, it just happily contradicts itself halfway through the sentence.
My pet theory is that these frontier models do quite a bit of brute force at the end - some kind of beam search - and silently downgrade you depending on demand or compute availability.
Still not great. This particular nugget is from Sonnet 5, default settings.
Not like adding glue to pizza. Here's an example from today (paraphrasing): "you need to run `git merge-base branch1 branch2`. Pay attention to the order of arguments, it is important: `git merge-base` is symmetric and returns the same value regardless of the order of inputs".
So which one is it? Symmetric or not? It's not even one of those where it self-corrects, it just happily contradicts itself halfway through the sentence.
My pet theory is that these frontier models do quite a bit of brute force at the end - some kind of beam search - and silently downgrade you depending on demand or compute availability.
Still not great. This particular nugget is from Sonnet 5, default settings.