kparaju

OpenAI see.

OpenAI ignore.

OpenAI train.

If Google lets LaMDA out in the open, they will see their margins shrink. Lose-lose for Google. Great move by Microsoft.

You could probably do something like (for django):

  python manage.py migrate

on pod startup. If there are no changes, it'll do nothing. If there are changes, it'll do the migration.

In the authors case, terraform will create the EKS (kubernetes) cluster, which then is responsible for creating the EC2 instances. The actual application containers are then created by EKS.

Thanks for a great post! It was super detailed and I loved reading it. I had a quick question about your pg setup. You mentioned that you use EBS for your persistence storage, which is locked by zone . You can't have an EC2 instance in Zone 1 mount a storage in Zone 3. Does this cause issues with your db? Especially as you have HPA and ClusterAutoscaler, your k8s nodes could be spun up in Zone 1 for pg autoscaling but your data is in Zone 3.

Some lessons I took from this retro:

- Disable autoscaling if appropriate during outage. For example if the web server is degraded, it's probably best to make sure that the backends don't autoscale down.

- Panic mode in Envoy is amazing!

- Ability to quickly scale your services is important, but that metric should also take into account how quickly the underlying infrastructure can scale. Your pods could spin up in 15 seconds but k8s nodes will not!

"It wasn't completely down because we had one rack in AZ A which was not impacted because we launched it to space on a Blue Origin rocket and forgot to remove the us-east-1 label."

FWIW, I don't think this YouTube outage has anything to do with the Instagram outage you are referring to.

All three of the big cloud providers have a solution for ML model deployment and support more libraries than just Fast.AI. What are some of the reasons one would use Deepserve.ai vs. the other cloud providers?

This is really neat though. The fact that this is so easy to use will be a big appeal to a lot of data scientists who don't want to write production code or deal with lots of bootstrap configuration. There are also a lot of benefits of abstracting the deployment as you can seamlessly add a lot of features like logging, or even make performance improvements by tweaking a few env vars and everyone will get it by default! Thanks for sharing.