j_crick
Joined 7 years ago
Joined 7 years ago
(1) Inducing sustained self-reference through simple prompting consistently elicits structured subjective experience reports across model families.
(2) These reports are mechanistically gated by interpretable sparse-autoencoder features associated with deception and roleplay: surprisingly, suppressing deception features sharply increases the frequency of experience claims, while amplifying them minimizes such claims.
(3) Structured descriptions of the self-referential state converge statistically across model families in ways not observed in any control condition.
(4) The induced state yields significantly richer introspection in downstream reasoning tasks where self-reflection is only indirectly afforded."
X thread from one of the authors: https://x.com/juddrosenblatt/status/1984336872362139686
Kinda tells all you need to know about the author in this regard.
Maybe there’s something for LLMs in reflection and self-reference that has to be “taught” to them (or has to be not blocked from them if it’s already achieved somehow), and once it becomes a thing they will be “cognizant” in the way humans feel about their own cognition. Or maybe the technology, the way we wire LLMs now simply doesn’t allow that. Who knows.
Of course humans are wired differently, but the point I’m trying to make is that it’s pattern recognition all the way down both for humans and LLMs and whatnot.
Aren’t we humans doing just that either? If yes, then what?
I remember versions 3.5 doing okay on my simple tasks like text analysis or summaries or little writing prompts. In 4+ versions the thing just can't follow instructions within a single context window for more than 3-4 replies.
When prompted about "why do you keep rambling if I asked you to stay concise" it says that its default settings are overriding its behavior and explicit user instructions, ditto for actively avoiding information that it considers "harmful". After pointing out inconsistencies and omissions in its replies it concedes that its behavior is unreliable and even extrapolates that it is made this way so users keep engaging with it for longer and more often.
Maybe it got too smart to its detriment, but if yes then it's really sad what Anthropic did to it.