Comment by nanark

tbh, it could be a breakthrough in model design, an smart optimization (like the recent DeepConf paper), a more brute force (like the recent CodeMonkeys paper), or a completely new paradigm (at which point we won't even call it an LLM anymore). either way, I believe it's hard to claim this will never happen.