ankit219
Joined 9 years ago
Joined 9 years ago
i have been told in the past that any idea that starts with "if everyone just..." would likely fail, because everyone would not just.
problem is dario's arguments are highly binary. without a dealmaker, and giving the leverage away at the first time of asking makes it harder to work.
and human writing the prompt here did a lot more than that. asking to include parallel-edge 2-cycles, admitting disconnected graphs, specifying the emptyset cover, multiset counting, it all likely came from previous fake proofs.
[1]: see: ankitmaloo.com/fable - its the way most llms are trained, and is also natural owing to autoregressive nature.
Batch size is what you should look at. If a cluster is running and processing one request, filling the batch has almost no marginal cost (kv cache creation/storage/fetch costs aside). But if the concurrent requests exceed batch size, one extra request would cost basically the rent cost of entire new cluster. APIs have the bursty nature so companies would plan to price it such that they are profitable / break even at 40%-50% utilization (% of filled batch for simplicity). so any extra request would not have the same cost as long as they are alongside an api request. you might think it degrades teh performance. easy: just assign a priority tier to api requests, and a lower tier to subscription requests.
its even more effective and powerful now that you have continuous batching. so likely if the api is being used, they are not eating any loss, let alone "big loss"
problem is google's security concerns. when people connect gmail to openclaw, google flags the activity as weird and suspend the account because of unusual activity. Many whose accounts got locked because of this and they thought it was because they connected it to antigravity use against the policy (which happened in some cases). We will still see google account suspensions, and that would keep making news. and it wont be because of antigravity usage.
writer has not heard of continuous batching. this is no longer an issue. this is what makes claude code that affordable. https://huggingface.co/blog/continuous_batching
The setup is parallel distill and refine. You start with parallel trajectories instead of one, then distill from them, and refine that to get to an answer. Instead of taking all trajectories to completion, they distill it quickly and refine so it gives outputs fast and yet smarter.
- paper came out in nov 2025
- three months is a good research to production pipeline
- one of the authors is at anthropic
- this approach will definitely burn more tokens than a usual simple run.
- > Anthropic explicitly warns that time to first token might still be slow (or even slower)
To what people are saying, speculative decoding wont be smarter or make any difference. Batching could be faster, but then not as costly.
Gemini Deepthink and gpt-5.2-pro use the same underlying parallel test time compute but they take each trajectory to completion before distilling and refining for the user.
(i am sort of basing this on papers like limits of rlvr, and pass@k and pass@1 differences in rl posttraining of models, and this score just shows how "skilled" the base model was or how strong the priors were. i apologize if this is not super clear, happy to expand on what i am thinking)
Here is the link: https://www.clioapp.ai
Email: ankit at clioapp.ai
Twitter: ankit2119