> Two of them knew they were extrapolating ("because Habsburg") and did it anyway.
You seem to imply that they ought not to. I disagree.
I wasn't familiar with the term before this post. Having learned it, were I given the task, I think I'd be strongly tempted to do the same extrapolation.
> If you're deciding which model to trust with instructions, "how much does it embellish beyond what I asked" and "does it behave deterministically" are directly practical questions.
Agency is agency. You still need to vet what the model's output is actually permitted to control.
You seem to imply that they ought not to. I disagree.
I wasn't familiar with the term before this post. Having learned it, were I given the task, I think I'd be strongly tempted to do the same extrapolation.
> If you're deciding which model to trust with instructions, "how much does it embellish beyond what I asked" and "does it behave deterministically" are directly practical questions.
Agency is agency. You still need to vet what the model's output is actually permitted to control.