So the TL;DR here is that Astra's trick is that its a turbo-charged weaker model vs a larger heavier one-pass model -- and the turbo is instead of reasoning by 'talking out loud' and generating intermediary steps, the reasoning is able to be stored (probably as KV) and re-run as purely without the generation of the text. Making it more effecient to run successively and I assume more intelligent as the act of turning the KV cache into lingusitics loses some dimensionality (especitally spacially)
double TLDR: This is a Turbo V4 instead of a huge V8 of a model.
double TLDR: This is a Turbo V4 instead of a huge V8 of a model.