Of course OpenAI used these guys' training data. This is part of their ratchet strategy: if you can monopolise the creation of new knowledge, you win. The moat they are building is user data, which a billion people are now throwing at them every day. They are solving difficult math problems using it. The only way to get the this data and these techniques back out is by distillation - expensive, slow work.
This moat ensures that Anthropic and OpenAI will remain at the frontier - no one else can train a super-intelligent model because they lack the trove of user data.
My main takeaway from reading this is - why do we always fall for this same trap? Every single time we allow a software company to build a monopoly in pretty much the same way. Microsoft, Oracle, AWS - They have a cool tech and we let them run away with it.
This moat ensures that Anthropic and OpenAI will remain at the frontier - no one else can train a super-intelligent model because they lack the trove of user data.
My main takeaway from reading this is - why do we always fall for this same trap? Every single time we allow a software company to build a monopoly in pretty much the same way. Microsoft, Oracle, AWS - They have a cool tech and we let them run away with it.