d-ramdass
Joined 6 years ago
Karma
40
Submissions
13
Member since
Sep 20
Recent submissions
Comment · 5 hours ago
Comment · 19 hours ago
I believe inference engineering is already an important emerging role which will continue to grow.
In fact today, CNCF at Kubecon announced a K8s inference engineering certification - https://cloudnativenow.com/kubecon-cloudnativecon-north-amer...
Our field guide helps folks get started in this area and the hope is that will be a valuable, practical resource for folks in their careers and work, as well as a valuable resource for folks using agents to help stand up their inference.
Comment · 21 hours ago
You can definitely use an agent to help Day 0 deployment but we found this information useful while operating a platform with SLAs for real users (see this series - https://www.upbound.io/blog/drinking-our-own-champagne-part-...).
Feel free to help contribute to the guide also!
Comment · 22 hours ago
That's fair. We tried to balance level of detail to make it approachable for newcomers to the space as well as practical tips for experienced practitioners. That being said, we'd love to incorporate your feedback in the guide - feel free to contribute to the repo and we'll also add those topics to our list to cover
Comment · 1 day ago
This guide is for platform engineers new to inference. If you need to understand how models and GPUs work together and how to run inference in production, this guide will help. You’ll also learn about the tools to use and how to grow into an inference engineering role. This guide was written by the maintainers of the open source CNCF sandbox project Modelplane
Comment · 12 days ago
We (the Modelplane OSS maintainer team) wrote an open guide on inference engineering based on our experience running open-source frontier models at scale with references to some of the best reading materials shared in the industry. Check it out, ask questions, discuss and contribute!
Comment · 17 days ago
It’s never been better to hill climb on problems especially deep systems problems. If you have the expertise and desire to carefully design goals, identify salient metrics, and recursively improve with rigor, modern AI can provide a massively powerful toolkit.
Comment · 3 months ago
I'm the 52x engineer :) AMA about how I burn my tokens
Not all of the nuance has translated into the guide as we tried to uplevel the specific learnings into transferable lessons and may have ended up sounding overlying generic in some parts.
We will improve it based on all the feedback we got (thank you!) and would really love to get the community involved to help keep this a valuable resource.