Except that he most certainly did! The bitter lesson isn't about LLMs vs specialized neural networks! It's about manually programmed expert systems vs machine learning. Alpha Go Zero isn't very good at anything but playing Go. It's a highly specialized brain only for that singular task. LLMs remain a problematic technology (both in terms of reliability and privacy) for a lot of domains and my own prediction is that smaller models that are much more reliable for a specific domain will become common after the utility of AI is known by way of LLMs, but the reliability or cost of running them makes it unsustainable.
Except that he most certainly did! The bitter lesson isn't about LLMs vs specialized neural networks! It's about manually programmed expert systems vs machine learning. Alpha Go Zero isn't very good at anything but playing Go. It's a highly specialized brain only for that singular task. LLMs remain a problematic technology (both in terms of reliability and privacy) for a lot of domains and my own prediction is that smaller models that are much more reliable for a specific domain will become common after the utility of AI is known by way of LLMs, but the reliability or cost of running them makes it unsustainable.