PhunkyPhil

I'm in no way familiar with LLM/ML architecture, but an N trillion parameter model is an enormous amount of training data... itself? When you predict a token you have the neurons/weights that were activated. Couldn't you run that backwards and put it in it's own training data?

I appreciate the consistency

How much issue do you take with animal testing in the rest of medicine? We already, more or less, torture an unfathomable amount of animals to get where we are.

I think it's definitely not as "cool", but it's almost a different problem class entirely since there's no correct answers here.

Knowing TCP connection protocols or deep compiler infra is impressive insofar as it's a mountain of knowledge that you've conquered and can stand on with a nice view, whereas "context engineering" and current AI engineering is a murky domain with no correct answers because... nobody alive knows the correct answers.

There's something different about wading forward into unexplored territory, or being able to establish techniques that nobody has attempted before. A lot of it will be noise or garbage (especially given a lot of it is LLM code), but I'd be careful to not mistake clever greenfield engineering with garbage simply because there's not a book written about it you can use as an oracle.

I understand the desire for something like this, but it's very naive. Different countries had unimaginably different histories which directly propagate to different cultures and values. *This is not natively bad thing*.

The problem arises when people bring in normativity when talking about these differences. It's okay to acknowledge that other people are different than you. This is a good thing. I don't want a grey homogenous world.

Yes, the theory is that they can bake in coding behavior in the weights that will introduce vulnerabilities...

I work at a company that holds very sensitive data and I don't think there will ever be a non-US-made open weight model in our walls.

You can't just look at the funding for math departments. I think the value generation potential of every mathematician in the world could exceed that of the capex of these companies. How much is "lost" when a brilliant mind goes into pure math research instead of nuclear engineering, or civil engineering etc. Cumulatively, a lot.

Edit: I'm fine saying we're spending too much money on AI, in fact I agree, but "theorems" are not the only output of these companies. It's almost definitely not going to be worth the expenditure, but the benefits are not going to be only lean proofs

> I want lower energy bills, lower rent, better understanding of health etc. more than I want theorems.

These efforts aren't mutually exclusive. I hate to be snide, but a lot of people would criticize you for being a mathematician because they want lower energy bills, lower rent, better understanding of health etc

How is this different than CPUs? I suppose in the last 5 years the architecture and tape has changed a lot as they move to make more LLM capable?

One of the problems is the power dynamic westernized/developed societies have. The article talks about just that: Australians brought shells in and basically one-shotted a rich culture for their own gain via hyper inflation.

If a missionaries message was basically "Believe in God and you'll have limitless shells", and backed it up with a crate of shells, they would all be converts.

The juxtaposition of someone's instinct to be thinking of bitcoin when reading about indigenous trade is at the best a bit funny, and at the worst small-minded when you consider the havoc those Australians brought onto their culture. Basically wiped out a foundational part of their way of life in an instant for their own gain. It's sad. That was my takeaway.

Depending on the age, people are spending between 4-6.5 hours a day on their phone. How much are people spending on computers? Their mattress? 2,000 for the cutting edge version is frankly reasonable for what a phone provides in this day and age, especially considering the last 3 years of inflation in the US...

These are just... obvious design choices that a foldable phone lends itself to. I'm not going to complain when Ford comes out with a new car that has a circle you turn when you want to turn the car. It's what works.

(a) identify any of their de-identified data that came from their usage of ChatGPT.

You don't need his login information, you just need to identify if anyone was approaching the NS problem using his method. Nobody else on earth (presumably) besides him, his team, and at best OpenAI were approaching the problem this way.