Comment by methodical

My position here is certainly debatable but IMO the pace of development in AI as a whole is actually pretty lackluster, and instead appears monumental because of the massive influx of VC dollars (and development of computing hardware!) which allows for these more and more monolithic models. Training larger and larger models doesn't really represent "development" to me in the sense I usually think of it. The largest hurdle with AI, and one that LLM has not and likely will never solve, is that it doesn't truly understand anything at all, but is just instead (in simple terms) a large matrix of inputs to outputs. The areas where current AI falls flat on its face are areas where it will never be able to encroach as it works today, no matter how many TPU hours you waste on learning.