ehsanu1

This confused me at first, so adding a tiny bit of context: The agents referred to here have nothing to do with AI Agents, and the linked report is from 2013.

Not directly relevant to the post being discussed, except as an example of how runaway automation can lead to unintended and large harmful consequences.

Surprised that usearch isn't in any of these, it's pretty fast.

Check this site for the first part? Maybe they can offer a notifications API, hah.

For my schemas, I found LLMs really wanted to just use markdown embedded in the strings, so I've been considering doing away with the schema. I also figure that embedding markdown in a string may make it perform worse as it has to juggle nested formats, and thus escaping and such (wanted: eval for this). By replacing the json tool call with basic markdown extraction, I'd lose some structured data but gain flexibility (html would be even more flexible).

Wondering if you are referring to adherence to required data in a schema when you say LLMs do better with json vs markdown, or something else? Or perhaps to tool calls and/or strict json output being more reliably formatted for clean extraction?

That a personal website? Prod means different things in different contexts. Even then, I'd be a bit worried about prompt injection unless you control your context closely (no web access etc).

I have. In fact, I've been building my own coding agent for 2 years at this point (i.e. before claude code existed). So it's fair to say I get the point you're making and have said all the same stuff to others. But this experience has taught me that LLMs, in their current form, will always have gaps: it's in the nature of the tech. Every time a new model comes out, even the latest opus versions, while they are always better, I always eventually find their limits when pushing them hard enough and enough times to see these failure modes. Anything sufficiently out of distribution will lead to more or less nonsensical results.

Business problems are essentially neverending. And humans have a broader type of intelligence that LLMs lack but are needed to solve many novel problems. I wouldn't worry.

An honest mistake.

What are the numbers? Are there problems other than context usage you refer to?

They aren't necessarily "stored" but they are part of the response content. They are referred to as reasoning or thinking blocks. The big 3 model makers all have this in their APIs, typically in an encrypted form.

Reconstruction of reasoning from scratch can happen in some legacy APIs like the OpenAI chat completions API, which doesn't support passing reasoning blocks around. They specifically recommend folks to use their newer esponses API to improve both accuracy and latency (reusing existing reasoning).

But those aren't things you can really separate for proprietary models. Keeping inference running also requires staff, not just for the R&D.

To spell it out for myself and others: approaching equivalent calculations for each individual attention block means we also approach equivalent performance for the combination of them. And with an error bar approaching floating point accuracy, the performance should be practically identical to regular attention. Elementwise errors of this magnitude can't lead to any noteworthy changes in the overall result, especially given how robust LLM networks seem to be to small deviations.

Could you substantiate that? That take into account training and staffing costs?