When we ran HPC CFD models at university we would just use a head node and SLURM to schedule work on machines. It worked really well. I'm not really sure what benefit Kubernetes adds in this case? If anything I feel like it adds more complexity.
Would love to hear from the OpenAI folks as to why they chose to use Kubernetes for this problem.
Would love to hear from the OpenAI folks as to why they chose to use Kubernetes for this problem.