trio8453
Joined 1 year ago
Joined 1 year ago
I wouldn't put too much emphasis on pre-training data, there are so many steps afterwards in modern LLM development that completely change the output. Generally, if you're looking at LLM output in 2026 and thinking something along the lines of "this must be what humans were doing with their language in the training data", you need to pause and reconsider, because modern LLMs are much more than machines which repeat what was in the training data.
Sometimes, not enough to believe you that you can do it better. And even if you could, to believe that it's overall worth it to work in that way.
I agree with you about simple languages though. I really like Go - the "there should be only one way to do things" from the Zen of Python actually applies to Go in a way that it doesn't apply to Python at all.
Keep it up as a hobby, find open source projects in which you can make meaningful contributions (that alone can be very hard), spend years in it, then maybe you'll have a better shot - whether all of that's worth it is for you to decide.
If I were you, I'd reconsider why you even want this in the first place.
1. Not train models to get better on certain verifiable tasks like math proofs?
2. Train models to get better, produce groundbreaking proofs and keep the results private?
3. ?