Comment

Comment by oersted

People sometimes forget that we had 2-3 years before ChatGPT of fairly high-quality LLMs (from Microsoft, Meta, Google), but they would have to take them down after a few weeks of being released because they became too controversial. Galactica is a good example for instance, there was also quite a good Twitter bot AI from Microsoft for a bit (forget the name), it got a bit racist and became infamous.

OpenAI's real achievement was to tame this technology, they were not the first to make good LLMs, but they were the first to fine-tune them with reinforcement learning to make them actually helpful and uncontroversial. ChatGPT was the breakthrough, not GPT-3. There's still work to do in this direction, it's a hard problem.

EDIT: by "success" in this context I am referring more to wide applicability and creating value as an actual product, whatever causes a project not to die in the real-world, I am very aware that from just the technical perspective the meaningful breakthroughs are different.

parent oersted 2 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton