To me, this entire piece reads like goal-post moving and lacks understanding of the field and was incredibly premature.
> I am confused as to why they continue to spend so much time on building Deep RL systems that beat games with clearly defined rules and point systems. I think everyone gets it now, with enough time, money, and computers almost brute forcing every single possible action, Deep RL can beat almost any game.
Right, before AlphaGo, superhuman Go was decades away. Then it became obvious and therefore easy. He brings up a laundry folding robot almost as if discrete games can be trivially applied to robotics. If you think about it for one second, Shogi, the game that AlphaZero solves with the largest action space has 10^5 actions. Consider a robot with 3 joints that can move in any dimension motors. Discretizing the continuous action space into 10 buckets yields an action space of 10^9. Something AlphaZero will never solve. In that sense, the entire analogy is a straw man and everyone in the field knows. This doesn't even touch the fact that 10 buckets would be useless for fine motor control.
Since they published this, there have been great advancements in learning action embeddings for robotic tasks. With a couple of expert examples, an algorithm can learn to open a door effectively. That's not that far from folding laundry.
Personally, my entire PhD thesis attempts to solves the exact deficiencies they raise. Simply put, the work goes on. Instead of saying that the field is useless, try to understand and contribute to it. At the very least, give technical reasons that it is a dead end path.
> I am confused as to why they continue to spend so much time on building Deep RL systems that beat games with clearly defined rules and point systems. I think everyone gets it now, with enough time, money, and computers almost brute forcing every single possible action, Deep RL can beat almost any game.
Right, before AlphaGo, superhuman Go was decades away. Then it became obvious and therefore easy. He brings up a laundry folding robot almost as if discrete games can be trivially applied to robotics. If you think about it for one second, Shogi, the game that AlphaZero solves with the largest action space has 10^5 actions. Consider a robot with 3 joints that can move in any dimension motors. Discretizing the continuous action space into 10 buckets yields an action space of 10^9. Something AlphaZero will never solve. In that sense, the entire analogy is a straw man and everyone in the field knows. This doesn't even touch the fact that 10 buckets would be useless for fine motor control.
Since they published this, there have been great advancements in learning action embeddings for robotic tasks. With a couple of expert examples, an algorithm can learn to open a door effectively. That's not that far from folding laundry.
Personally, my entire PhD thesis attempts to solves the exact deficiencies they raise. Simply put, the work goes on. Instead of saying that the field is useless, try to understand and contribute to it. At the very least, give technical reasons that it is a dead end path.