tevlon
Joined 11 years ago
Joined 11 years ago
Impressive work, indeed. vermiceli the GOAT.
For AI usage: It was all done by Claude/Codex, but it wasn't that straight forward. First: i started in April and the models weren't good enough and would make errors. You could see those immediately in glitches. Then, i spent some time in building a harness and collect telemetry data in Mesen. Once, i had that: i just played the game in the emulator and asked Claude Opus 5.5 to use this as verification. The rest was smooth.
I was able to reproduce the results of the original gpt-1 paper with my gaming PC. I don't even have alot of VRAM. My NVIDIA GeForce RTX 2060 SUPER was able to reproduce most of the results with just 1 hour of training. I would totally recommend to do the same, if you are interested in pre-training LLMs.
The code is here: https://github.com/epoyraz/modded-gpt-1 But, you can also just ask Claude 4.8 or Codex 5.5
This is one of the reasons people love using xona: no subscriptions :)