karterk

Where is this, if I may ask?

> The moat of the frontier folks is just compute.

This is not really true. Google has all the compute but in many dimensions they lag behind GPT-5 class (catching up, but it has not been a given).

Amazon itself did try to train a model (so did Meta) and had limited success.

Satya saved Microsoft by doubling down on Azure and Cloud. Something that Balmer failed to do with Mobile.

There are many different classes of problems that are affected by tokenization. Some of them can be tackled by code.

It's interesting how all focus is now primarily on decoder-only next-token-prediction models. Encoders (BERT, encoder of T5) are still useful for generating embedding for tasks like retrieval or classification. While there is a lot of work on fine-tuning BERT and T5 for such tasks, it would be nice to see more research on better pre-training architectures for embedding use cases.

> Cows are treated very well in all the ranches I've been on.

Unfortunately the entire premise of your argument is based on a personal anecdote that's a gross generalization of vast number of cattle farms.

For JIT-ing you need to know the sizes upfront. There was an experimental branch for introducing jagged tensors, but as far as I know, it has been abandoned.

Can you give some examples of problematic C idioms in C++?

Every billion dollar idea is a million dollar idea.

Every million dollar idea is a 100K dollar idea.

There are always smaller segments within large segments that you can go after.