I would add to the list a few questions about the evolution of cost dynamics going forward, given the advent of new sequence modeling architectures.
As we all know, Transformers are very expensive to train and run because their compute cost is quadratic in context length: O(N²).
If newer model architectures like RWKV, Mamba, and various others, which incur cost that is linear in context length, O(N), prove as successful as Transformers, the demand for compute for a query of N tokens would decline from O(N²) to O(N).
For a sequence with N = 100,000 tokens, it would mean cost dropping by a factor of 100,000×. That's not peanuts!
The implications for all market participants would be significant.
As we all know, Transformers are very expensive to train and run because their compute cost is quadratic in context length: O(N²).
If newer model architectures like RWKV, Mamba, and various others, which incur cost that is linear in context length, O(N), prove as successful as Transformers, the demand for compute for a query of N tokens would decline from O(N²) to O(N).
For a sequence with N = 100,000 tokens, it would mean cost dropping by a factor of 100,000×. That's not peanuts!
The implications for all market participants would be significant.