Nice summary. Reading this reminds me about the strong encryption discussion.
> We optimize what we can measure, not what we actually want to achieve. We hope and pray that these are the same thing, but they often aren’t.
He points out the core problem with LLMs. I believe it is impossible (or extremely expensive) to ensure that the models are aligned safely for everyone and any intention. And 'safe' can mean different things for a different audience.
> We optimize what we can measure, not what we actually want to achieve. We hope and pray that these are the same thing, but they often aren’t.
He points out the core problem with LLMs. I believe it is impossible (or extremely expensive) to ensure that the models are aligned safely for everyone and any intention. And 'safe' can mean different things for a different audience.