jsharf

If you have independent copies of the network learning gradients, then you’re effectively making the batch size smaller— unless you’re doing an all collect and making them sync, in which case there’s a lot of overhead

When you take a batch and calculate gradients, you’re effectively calculating a direction the weights should move in, and then taking a step in that direction. You can do more steps at once by doing what you say, but they might not all be exactly in the right direction, so overall efficiency is hard to compare

I am not an expert, but if I understand correctly I think this is the answer.

I want to also mention that the previous model was 3.7. 3.7 to 4 is not an entire increment, it’s theoretically the same as 3 -> 3.3, which is actually modest compared to the capability jump I’ve observed. I do think Anthropic wants more frequent, continuous releases, and using a numeric version number rather than a software version number is their intent. Gradual releases give society more time to react.

He's not saying that all non fiction is bad, just that the incentives are misaligned, and to be fair at least in my experience, there are a lot of popular non-fiction books where each chapter is repetitive, and I feel the whole thing could have been written in 2-3 chapters, if publishing a 30-page nonfiction book wasn't taboo

Wow, you can refocus the direction after the audio is recorded!

This would be cool to mix with VR, so you could hear different conversations as you move around a virtual room

I assume it’s easier to find an engineer who went to engineering school to learn how to build airplanes that are safe than it is to find an MBA who went to business school to learn how to build planes that are safe. (It’s not about the knowledge but about the root desire)

Similarly, I assume it’s harder to find an engineer who went into the field purely for money.

I do think on average engineers will prioritize safety (since they likely understand failure modes and production and long tail statistics better. We literally have to take engineering ethics classes), at the cost of doing a worse job at running the business. But when the business requires this level of safety, that IS doing a good job.

I think planes can still fly with the rudder loose? If the bolt falls out and it loses control, wind will push it into the neutral position and then flying will still be possible with other control surfaces? But I guess if the pilots don't know and it happens suddenly at a critical moment or if the bolt causes the rudder to get jammed, then that would be really bad. But I assume it falling out would result in the rudder loosely returning to neutral...

If we have intelligent AI that can automate programming, then making really good robots will not be a problem. While not trivial, actuators and power systems are not the reason why we don’t have robots that can do all manual labor for us. Software is the reason, and the same kind of software that’s learning to code (machine learning) can also be adapted to washing dishes, folding clothing, doing craft labor or previously human manufacturing jobs.

Accelerating programming and information jobs also means accelerating the creation of robots that can do these trade jobs

Recommend passing the speech-to-text narration through a round of GPT4 API to correct for any transcription errors (use some prompt giving context that it's speech to text)

Suggests there's other variables involved, like time of day taken, other supplements taken simultaneously, metabolic processes, diet, and maybe even the placebo effect.

Well, assuming they likely have some PFAS already in their system, you might just be giving them blood with the same concentration of PFAS that they already have.

They need to be dimensioned and finished quite accurately. Most plywood has a slight bend to it. At the speeds and accelerations needed to be competitive, any imperfection would mean your car will fly off the track or hit a wall.

Wait isn’t PPG for heart rate?