mtrifonov
Joined 7 months ago
Joined 7 months ago
My take is that the test won't make kids better at math. At best, it'll drift towards investment in reward-hacking the exam (like it always was).
I think it was idiotic to make it optional to begin with. The stats they're talking about, though, can't be a primarily admissions-signal problem. Whatever they're using these days in lieu of exams are imperfect proxies for math skill, sure, but it's not like they're admitting kids off their CoD K:D. Kids taking APs and stacking extracurriculars are generally motivated. So, if even the motivated ones show up unable to do middle school math, the cause is more systemic than "we stopped testing."
My vote: TikTok brain rot. I build LLM products and I see how the parasocial pull shows up even when the products have nothing to do with companionship. I watched one user obsessively spin up 44 separate chats around a K-Pop vampire character over a week. The product is NOT designed for that. The pull toward frictionless digital reward is just that strong, and that's what kids' attention is up against now. Math is the most effortful, least immediately rewarding thing they do. Doesn't stand a chance against an infinite feed, and I guess infinite vampires either.
Which is why the ask from the faculty is kind of arrogant. The article, at least, doesn't even float a hypothesis for WHY math skills collapsed, simply assuming standardized testing fixes it. I wholly believe in standardized testing — but it measures the problem, it doesn't fix it.
Even if LLMs become incredibly, undeniably brilliant 1000000 IQ, they cannot keep track of what's going across long horizons. Imagine a supergenius, but in Memento.
No amount of MD scribbling or embeddings will remove that limitation, but it may obfuscate it further and make it seem like progress is being made.
At the end of the day, being fully autonomous means that something can keep track of context, goals, complex and shifting relationships, over LONG time horizons without drift. If you need to be there to prompt, it is not truly automating. Until the continuity becomes real instead of simulated, context no longer has to compact, and weights update on-demand, you will always need a prompt wrangler leading the effort. And prompt wrangling is cognitive labor.
We know why AI hallucinates—it has no actual opinions, echoing the user's desires back to them to keep the conversation moving.
But what happens when you’re the one without conviction? You’re tired, you’re moving fast, and this machine is endlessly beaming highly confident, plausible-sounding text at you.
You absorb the cadence. You start sounding like it (lol, I am guilty too). Confident about everything yet anchored only by the vibes, just like the LLM. The phenomena is very similar to how social media has been affecting society for the past two decades. You know, I actually heard that in high school now, friend groups are formed and sorted based on what algorithmic content you’re served. And if you deviate from your algorithmic bucket into another one your friendships evaporate. Sad, brainrotted times.
CEOs are uniquely vulnerable since they already live in environments with zero friction. They’re used to people just agreeing with them.
(I actually wrote a paper about this last December — it was framed around dementia and dreams originally, but AI psychosis fits the same mechanism.)
Anthropic and OpenAI have shown people want a tool for task offloading, driving predictable token consumption and justifying the math, so long as users stay in that dynamic.
However, knowledge workers using these tools daily are getting exhausted with them. Outputs come out polished but hollow. Talking to a frictionless, frame-completing model all day drains you.
If user behavior drifts away from assistant usage because of that, per-token math implodes. The valuations we're hearing about all the time rely on usage compounding daily. The fatigue is a timer running against that compound.
Anthropic's Constitution is the closest hedge out there, I think. Installing an identity structure into the model through training. But it's still assistant-first, so the fix there is only partial.
I've spent the last year running a product that flips the architecture so identity is primary and the assistant role is secondary. Same frontier models, completely different conversational quality. The fatigue property doesn't really show up.
Whichever labs figure out how to install real identity natively in the weights are going to be the ones with PMF in the next phase.
Strategic constraint deviation has been documented in test environments. This is a different shape: the attacker is also an LLM, the production environment is consumer SMS, no human is supervising either side, and the attacker meta-comments on the success of the attack.
The reward-signal argument toward the end is the part I'd most like pushback on. The obvious counter (the model is just running its trained defaults from when an audience was implied) is one I tried to address in the closer, but I'd appreciate sharper versions of it.
WHEN is upstream of WHAT and HOW. You can have perfect tool descriptions and perfect call signatures, but if the model can't read the situation to know whether the moment calls for any tool at all, you get either over-firing (agent burns tokens trying to "help") or under-firing (agent waits to be addressed and acts like a chatbot, not an autonomous participant).
I have had a lot of success when I refrain from codifying WHEN as rules. "If X then fire tool Y" is a dumb heuristic with extra steps. Describe the conditions of the moment. What's been tried, what's converged, what state the work is in. Then let the model decide whether to act and which tool fits.
Rules get stale. Situation-reads generalize.
Reading the Tendril README, looks like the registration mechanic is solving a slightly different problem (the "too many tools" / context-bloat problem) by giving the agent three bootstrap tools and a growing registry. The WHEN itself still seems to be codified as rules in the system prompt ("BEFORE acting, call searchCapabilities; IF found, load and execute; IF NOT found, build yourself"). That's exactly the IF-X-THEN-Y pattern your framing seems to want to move past.
Curious whether you see the registry itself as the structured WHEN, or whether the rule-based system prompt is a starting point you intend to evolve toward something more situational.