Comment by choppaface

It will be interesting to see how well this scales. I have seen AWS EFS fall over under heavy read load (lots of reads for the same TB-scale data). A lot of poor souls using Horovod with no data team use EFS to “distribute” data in data-parallel SGD.