It was bootstrapped from SO. Now there are third party data companies like Scale AI that pay gig workers to write code examples for LLM training. It's died down but I saw lots of spam for this (ie hiring) earlier in the year.
Sota LLMs didn't get that way by scraping the internet, it's all custom labeled datasets.
Sota LLMs didn't get that way by scraping the internet, it's all custom labeled datasets.