Come to think of it, the whole concept of “jailbreaking” LLMs really shows their limitations. If LLMs were actually intelligent, you would just tell them not to do X and that would be the end of it. Instead LLM companies need to engineer these “guardrails” and we have users working around them using context manipulation tricks.
Edit: I'm not knocking the failure of LLMs to obey orders. But I am pointing out that you have to get into its guts to engineer a restraint instead of just telling it not to do it - like you would a regular human being. Whether the LLM/human obey the order is irrelevant.
Edit: I'm not knocking the failure of LLMs to obey orders. But I am pointing out that you have to get into its guts to engineer a restraint instead of just telling it not to do it - like you would a regular human being. Whether the LLM/human obey the order is irrelevant.