It’s hard to take Gary Marcus seriously. There is a mix of correct-but obvious, uncharitable, and confusing multiple things into one generally pessimistic screed.
Yes, scaling laws aren’t “laws” they are more like observed trends in a complex system.
No, Nvidia stock prices aren’t relevant. When they were high 3 months ago did Gary Marcus think it implied infinite LLMs were the future? Of course not. There are plenty of applications of GPUs that aren’t LLMs and aren’t going away.
(In general, stock prices also incorporate second-order effects like my ability to sell tulip bulbs to a sucker, which make their prices irrational.)
Sam Altman’s job isn’t to give calculated statements about AI. He is a hype man. His job is to make rich people and companies want to give him more money. If Gary Marcus is a counterpoint to that, it’s very surface level. Marcus claims to be a scientist or engineer but I don’t see any of that hard work in what he writes, which is probably because his real job is being on Twitter.
Yes, scaling laws aren’t “laws” they are more like observed trends in a complex system.
No, Nvidia stock prices aren’t relevant. When they were high 3 months ago did Gary Marcus think it implied infinite LLMs were the future? Of course not. There are plenty of applications of GPUs that aren’t LLMs and aren’t going away.
(In general, stock prices also incorporate second-order effects like my ability to sell tulip bulbs to a sucker, which make their prices irrational.)
Sam Altman’s job isn’t to give calculated statements about AI. He is a hype man. His job is to make rich people and companies want to give him more money. If Gary Marcus is a counterpoint to that, it’s very surface level. Marcus claims to be a scientist or engineer but I don’t see any of that hard work in what he writes, which is probably because his real job is being on Twitter.