neversupervised

Im sorry to hear about your situation. Have you blogged about it? I’m curious about how the volume of your type of work has been changing over the past 2-3 years.

This just feeds a certain narrative and allows people to take exactly the wrong conclusion. Just because there’s some uncertainty at the edge, it doesn’t change where things are going.

Years of experience don’t correlate to output in all careers. Surgeons and engineers get better over time. This might not be true for all jobs. Meanwhile, management is naturally capped because every manager necessarily needs people to manage under them, so there’ll be 1/N^y managers at the yth level of the org. Unless loyalty ought to be reward for its own sake, it’s not clear why 100% of workers should get promoted indefinitely.

This is not how people use LLMs. If you ask one of these questions you’d get a longer answer, often grounded on the internet. I speculate that conditional on a smart human operator interpreting the results, such interpretations across vendors converge more often than this report makes it seem.

This is just a glitch in time. It’ll be agents talking to agents. We won’t be able to keep up.

Check out Pangram

It’s completely wild to me that lifelong programmers come into contact with agentic coding and come to the conclusion that their jobs are safe for one reason or another. AI will definitely be able to write entire software, inclusive of figuring out requirements and asking the right questions. It’s not that far already. Why is it that everyone looks at weaknesses of a technology that didn’t exist a couple years ago instead of appreciating the incredible rate of improvement? I know why, because it’s inconvenient to the narrative of what makes us valuable. But still, our job is to turn ideas into a sequence of logical steps. Why can’t we do the same when forecasting the impact of AI on our jobs?

This is a great way to become well-versed in benchmarking.

Terminal Bench is the future

But this is the good kind of goalpost moving

That paper focuses on breaking the harness, the same hack applies to all tasks. Here we are breaking tasks individually. If these were put on a different, more secure harness, most of the exploits would still work.

This is nonsense. I’m sorry. AI will completely upend the workplace and the economy. Whether that’s self evident today in the numbers in the way that we track those numbers, which is based on how things have historically worked, is not relevant. First principles thinking is enough.

C’mon. Stop wishing for a future that feels convenient. This is not the world in which we live. Everything will change. Let’s help people accept and react to that.Let’s stop with the comfort talking and false hope.

I've been a contributor and reviewer for terminal bench since last August, and this post is about what I've learned designing and reviewing tasks. The guidance is broadly applicable to anyone building an agentic benchmark.I would love feedback from the HN community.