timshell

I think many cognitive scientists think language is a core part of human cognition, and that LLMs were a major advance in AI/cognitive science due to connecting AI algorithms with language production, but LLMs aren’t considered (or even aiming) to be accurate models of human cognition

This was the core premise behind my research during grad school and beyond. My favorite project — which ultimately became my dissertation — was a proposal to connect reinforcement learning algorithms, neuronal data, psychological behavior, and phenomenology. Specifically, the idea was the feeling of cognitive fatigue is a mathematical signal to ‘rest’, in which the value of rest is defined by offline mechanisms that contribute to decision-making. I proposed a connection to hippocampal replay, and was able to leverage fMRI and choice data to evaluate the hypothesis.

Zooming out of my specific work, the “best” success stories were connections across math/CS, neuro, and psych and the golden child is the dopamine model of reinforcement learning where firing patterns encode the TD prediction error

> In that case you could just add a login form

This is the product insight. We're not going to deploy Stroop tasks for authentication :)

Thanks all for the discussion! Would like to highlight two parts that maybe didn't come fully through, and we'll work on making this clearer:

1. CAPTCHAs can still detect AI agents...if you know where to look. Most commercial CAPTCHAs are not doing the cognitive process tracing you see in our paper. Nor are they really doing 'behavioral biometrics' (but that is slightly tangential here). Our CAPTCHA example here is about repurposing the current paradigm with a new methodology (cognitive process tracing) in a way that is able to combat human/machine discrimination in a way that's independent on frontier AI progress.

2. There are lots of concerns about adversarial robustness, which are very fair, and we reported some fine-tuning tests in the paper. Generally, there are two mental models for me that work, both framing fraud as an economic game.

First, compare AI spoofability concerns to something like a passport or a fingerprint. The cost to mimic continuous cognitive and behavioral patterns over time seems more computationally complex. In other words, sure this method is not bulletproof with infinite resources, but nothing is. We rely on defeasible mechanisms everyday, and our job is to make that significantly securer.

Along these lines, there's a common line of criticism that suggests once fraudsters know the game, they will solve the game. The CAPTCHA presence in the 2000s didn't mobilize massive deep learning / image recognition advances from the fraud community. Nor are these same bot farms solving quantum computing despite there being immense incentives to. If anything, the real threats are stuff like JavaScript injections, not really fully simulating human cognition

This is relatively close to our conclusion from the paper: unless agents are specifically trained for the task and know all the information ahead of time, they're not able to generalize from one cognitive CAPTCHA to another