Comment by lbrito

Forgive my ignorance, but if the corpus of coding data was always 90% bad, isn't that the same data being used for training LLMs? How are they magically any better than that average?