I agree that ML on Spark was only a limited hit—- iterative jobs would actually be feasible versus Hadoop—- I still have yet to find a better ETL and SQL tool, and that’s a big part of most ML projects.
I’m worried about Ray as a SAAS Co because so far it looks to me like they’re riding reinforcement learning hype. They’d need to really penetrate the users of Horovod and Tensorflow Distributed to get beyond a beach head. And what if TPUs and Cerebras become more common? Because then the maker for multi-machine workloads becomes smaller (definitely not zero though).
I’m worried about Ray as a SAAS Co because so far it looks to me like they’re riding reinforcement learning hype. They’d need to really penetrate the users of Horovod and Tensorflow Distributed to get beyond a beach head. And what if TPUs and Cerebras become more common? Because then the maker for multi-machine workloads becomes smaller (definitely not zero though).