Joined 3 years ago
Everyone is giving agents their own sandboxes these days, so that part isn't new. But I wanted to share the "why" and what we learned from building them. We first built them the way you build anything else in the cloud, but agent sessions don't behave like cloud jobs and a lot of cloud habits were wrong for them.
A few examples:
1. We first built a microservice that was responsible for managing each agent session, and thought we could scale it horizontally. That doesn't work, not if you want to give your agents a real file system and tools that need to be installed on that machine. We later gave each machine its own sandbox.
2. Our agents power our UI, but also live in Slack, Github threads and wait for people to reply. Our sandboxes had a 1-minute timeout and a cold-start. So once a sandbox died, it took anywhere from 10 to 30 seconds to spawn, because of which the users had to wait longer before they received a response from our AI on the UI or in the threads. Now, we maintain warm pools of sandboxes which can be acquired by a session in milliseconds and each sandbox is kept alive for 5 minutes after each reply.
The post is fairly high level, but if you want more detail on any part of it, I'm happy to talk about it here.
Location: India (GMT +5:30) - used to working in EU & US hours
Remote: Yes
Willing to relocate: No
Technologies: TypeScript / JavaScript, ReactJS, Node.js, Next.js, Redux, MobX, Angular, HTML, CSS, SCSS, TailwindCSS, Python, Django, Flask, Fast API, Go lang, Gin, Kubernetes, Docker, most of AWS Cloud, Redis, Azure, HashiCorp Cloud, Terraform, MySQL, PostgreSQL, MongoDB, DynamoDB, CosmosDB, Cassandra, Java, Spring Boot etc. (More details in my resume)
Résumé/CV: https://drive.google.com/file/d/1Kr7MmTc3FAN7a-2xV_pttmjiFoRAAFon/view
Email: hello AT atchyut.dev or pulavarthi.preetham AT gmail.com
Website: https://atchyut.dev
Current designation: Senior Engineer (Technical Lead)
Current org: Postman
LinkedIn: https://www.linkedin.com/in/atchyutpulavarthi Location: India (GMT +5:30)
Remote: Yes
Willing to relocate: No
Technologies: TypeScript / JavaScript, ReactJS, Node.js, Next.js, Redux, MobX, Angular, HTML, CSS, SCSS, TailwindCSS, Python, Django, Flask, Fast API, Go lang, Gin, Kubernetes, Docker, most of AWS Cloud, Redis, Azure, HashiCorp Cloud, Terraform, MySQL, PostgreSQL, MongoDB, DynamoDB, CosmosDB, Cassandra, Java, Spring Boot etc. (More details in my resume)
Résumé/CV: https://drive.google.com/file/d/1Kr7MmTc3FAN7a-2xV_pttmjiFoRAAFon/view
Email: hello AT atchyut.dev or pulavarthi.preetham AT gmail.com
Current designation: Senior Engineer (Technical Lead)
LinkedIn: https://www.linkedin.com/in/atchyutpulavarthi
I've been a software engineer for sometime now, and had so many people I know ask me how to get into tech, and I recommended some to simply learn HTML, CSS, JavaScript and then React, Next.js or Node, Express etc. to land their first frontend or backend role and figure things out from there, but in 2024, I think that's not enough, a lot of companies are actively choosing not to hire new grads or new programmers, and I feel people learning to code should learn to 'learn' instead of just learn to code. Hope this guide helps someone.