chrsw

I'm pretty sure I heard Chen say on a podcast that he has a significant backlog of posts that he puts up when he feels like it.

The modern internet is a largely centralized network built for the following purposes:

- commerce

- surveillance

- platforming voices for states and the wealthy

I see. So maybe the situation isn’t as bad as I thought. I was thinking if you took all the money and resources spent on building the biggest models, across the two big labs, the hyperscalers, the dozens of neo labs, and you reallocated that money to building value on cheaper Chinese models that would be a much better proposition than doing whatever it takes to be head of Chinese model performance by a few percentage points for a few months. And I wasn’t just referring to the cost of training one model, I was considering the total cost of maintaining an entire ecosystem with the ability to do this.

But what I’m hearing is this is a negligible cost compared to inference buildouts. Fair enough.

Maybe I'm conflating. I could be dead wrong about all this. But I'm trying to zoom out and look at the big picture. So, I'm letting the details sort of blend away and I'm just asking myself: "Is such a large financial sink in our economy, at the macro level, really going to offer the value that the investment warrants?". So, the particulars of who exactly is spending what where I guess doesn't concern me as much. What I'm thinking about is American companies, American people, American dollars going to something that seems like an iffy value proposition.

Now, if you tell me American businesses are turning cheap Chinese tokens into goods and services of high value, for Americans and people all over the world, then obviously I would have a different outlook on what's happening here.

So, the US economy spends trillions of dollars to be at the "frontier" and China puts out a model that basically does the same thing, for free, and for a tiny fraction of the development cost. How is this sustainable?

My prediction is that in in a few years, maybe more, the current models will be seen as so primitive that they're not worth mentioning outside of a historical context.

I can think of plenty of tasks that even the closed frontier models fail miserably at. From my perspective we have a long way to go for these models to get better.

There are bills to pay. Who has time for friends?

I don't understand how people are so certain on how this will play out. We have people basically saying companies like Anthropic offer no value because open source models are free. Then we have others saying once frontier AI labs go RSI FOOM takeoff they'll essentially _become_ the tech industry.

None of this sounds credible.

A hybrid approach might work better for robotics. A model-in-silicon for body control and sensing, then a different model on processors for reasoning, speech and decision making.

No, there still plenty of tasks that LLMs can’t do well or at all. Yes, we should absolutely make models cheaper but there’s a long way to go until they’re “good enough”. And I’m talking about the frontier closed models too, not just the open ones. So it’s not an “instead of” position we’re in. It’s both.

Imagine if a car company like Ford or Toyota came out and said their new cars are kind of out of control and they needed special indemnities and protection from liabilities if they want to keep selling cars.

You can use open weights models right now and ignore all this fuss. And by the time open weights models are objectively and subjectively on par with what Fable/Astra are today, the frontier labs will probably offer something newer, more capable and more expensive. The models OpenAI and Anthropic roll out clearly aren’t for people on a strict budget.

Maybe one day open weights and closed models will hit the same wall?

Would a 100% compatible and open source CUDA stack be “competition” too?

We need to immediately set the precedent that ultimately humans and companies are responsible for what their AI systems do.