Comment by noemit

i think its more efficient for models to write queries than read large(ish) documents. at some point you have to have a tradeoff - smaller docs, but the model has to find them (thats a query) or larger docs, and the model has to read them.

My take is AI native apps that have longevity will prioritize token-efficiency - and I believe queries do that.