I think there's an interesting disconnect right now between research and practice. Cutting-edge research does feel like it's reaching a plateau - across most AI fields even "major" breakthroughs are only gaining a couple percentage points and we're probably starting to hit the limits of what current approaches can achieve. When the state-of-the-art is 97% on a task, there's only so much room for improvement. Yoav Goldberg posted a tweet about Facebook's RoBERTa model that summed it up pretty well: "oh wow seems like this boring public hyperparameter search is going to take a while" [1]. There's a vague feeling of "What's next?" now that all the benchmarks are fairly well-solved but AI in general clearly doesn't feel solved.
However, state-of-the-art models aren't really used in production yet. I think the trend of "use AI/ML to solve X" has only started to pick up in the past 2 years, and it'll continue well into the 2020s. The process of taking research models and putting them into production is not standardized yet, and many models don't even really work in production - if your model takes a second to do an inference step that's fine for research but maybe not for a real product.
I think in the next decade, on the research side, benchmarks will be beaten less often, and instead there will be more focus on trying out radically new things, understanding weaknesses in current techniques, and finding new measurements that assess those weaknesses. On the industry side, there will still be lots of cool and exciting new achievements as already-known techniques are applied to old problems that haven't been addressed by AI yet.
As an aside, this was the first time in my life that I read the phrase "10s" referring to the 2010-2019. Kind of an odd-feeling moment!
[1] https://twitter.com/yoavgo/status/1151977499259219968
However, state-of-the-art models aren't really used in production yet. I think the trend of "use AI/ML to solve X" has only started to pick up in the past 2 years, and it'll continue well into the 2020s. The process of taking research models and putting them into production is not standardized yet, and many models don't even really work in production - if your model takes a second to do an inference step that's fine for research but maybe not for a real product.
I think in the next decade, on the research side, benchmarks will be beaten less often, and instead there will be more focus on trying out radically new things, understanding weaknesses in current techniques, and finding new measurements that assess those weaknesses. On the industry side, there will still be lots of cool and exciting new achievements as already-known techniques are applied to old problems that haven't been addressed by AI yet.
As an aside, this was the first time in my life that I read the phrase "10s" referring to the 2010-2019. Kind of an odd-feeling moment!
[1] https://twitter.com/yoavgo/status/1151977499259219968