spindump8930

That's overly generalized. Perhaps they became a buyer after google stripped top staff, but before that they were certainly a producer, and had many advances in cutting edge inference at the time.

"Commits are to private repos" why is this unusual? Very expected if OpenAI has private repos in a github org.

They hired many accomplished researchers, with track records of both papers and work at other frontier companies. They gave those folks huge autonomy, huge budget, and 18 months - some would say at the expense of the rest of the company. Check out blind references to TBD. I'd call the strategy ruthless, but it certainly was effective.

Strong agree, a good way forward is letting user agent once again truly mean an agent on behalf of the user. And speaking anecdotally, I've seen different traffic "shapes" under the same UA, which leads me to suspect that some actors might be disguising traffic as more mainstream labs.

Well, potentially very little! It could be subscription usage, or just really inefficient code authored by an agent. It probably costs LESS in tokens to write a poor quality cloner that breaks upstream things.

The problem (and contrast with other approaches) is that mat muls requires synchronization. Arranging your networking and training structure to maximize compute and minimize communication is the main craft of ML training infra folks. In your example, yes you can compute layers on different machines (i.e. Tensor Parallelism), but you must be very careful in how you arrange it.

GPU failures are frequent enough that at a certain scale, you constantly have workers dropping out. Designing systems that can still keep training is very interesting!

All of those are good references. Other folks in the thread are missing distinctions between pre training (~the internet + curated sources) and post training (~instructions and RL)

Reflection isn't really "from no where", they have huge financial backing, 10K of the latest GPUs, and many ex leads from the established labs.

There are many measurable metrics and I don't see any that seem impressive. Care to share the ones you found exciting?

Thanks for what you've done and continue to do for the ecosystem!

Yes, but even nvidia doesn't have unlimited gpus. Everything they hold for internal teams is a reduction in revenue and customers can have aggressive commitments.

This is one of the jagged mismatches between users and LLM developers. The median user doesn't care or want to know about static models and knowledge cutoffs and whether a model can do tool calls or if tool calls even happened. They just want something that works.

Fortunately increased capabilities seem to make this a basic expectation with new releases.