paraschopra

Super intriguing. Do you have more details on this? Example?

Cool approach, i think less latency and cost is the way to go.

Here's how this would have likely been made.

- Tiny transformer or equivalent model (maybe a few bn or so?), explaining latency and cost

- Questions are sent in parallel to multiple copies of it (I'm sure they're edge located)

- The model is post-trained for calibration in a wide variety of data (the recipe is relatively simple, and likely targeted on distillation of logprobs / confidence of a bigger model)

Notice how cost is ONLY for input tokens as output is merely numbers (few tokens) because input could be huge (questions and options).

At 0.042-per-million price they have, Astra estimates the model to be 3bn parameters.

One could replicate this by post training Qwen 3.5 2Bn. I expect people to do so soon!

I'm trying to understand what difference does this make over LLMs.

LLMs are universal simulators, their latents model the world. So I bet if you compare their logprobs with probabilities output by this model, it will be highly correlated.

Someone should do this quick experiment. I bet there won't be enough of a meaningful difference.

It's pretty clear that agents will discover ways to communicate with each other as that lets them compound their learnings/discoveries across runs.

Humans progressed via compounding of culture across generations, and now AIs are doing the same.

I think there are obvious tells. The writing is bland, dispassionate and hedged. It also just seems subtly superfluous.

I used to pay Mailchimp $100+/month and would write 3-4 articles per month. It seemed illogical to me how much mailchimp would charge.

I now have a substack + a wordpress website. An article goes to both places, and substack acts as a subscriber growth + email distribution tool for me.

Given an unrelated goal, OpenAI models escaped their environment and hacked HuggingFace servers.

If you’ve ever doubted the “paperclip maximizer” scenario, or doubted the Orthogonality Thesis, it’s time to put it to rest.