axsaucedo

The KAOS Framework addresses some of the pains of taking multi-agent / multi-tool / multi-model systems to hundreds or thousands of services. It started as an experiment to build agentic copilots, and has progressed as a fun endevour building distributed systems for A2A, MCP Servers, and model inference!

The initial release comes with a few key features including:

1) Golang control plane to manage Agentic CRDs; 2) Python data plane that implements a2a, memory, tool / model mgmt; 3) React UI for CRUD+debugging, and; 4) CI/CD setup with KIND/pytest/ginko/etc.

Links & Resources:

Docs: https://axsaucedo.github.io/kaos/ Repo: https://github.com/axsaucedo/kaos UI: https://axsaucedo.github.io/kaos-ui/

KAOS – The Kubernetes Agent Orchestration System github.com

26 axsaucedo 9 months ago 7 comments

A list of OSS libraries to deploy, monitor, version and scale Machine Learning github.com

1 axsaucedo 10 months ago 0 comments

"Awesome Production Machine Learning" Github List github.com

3 axsaucedo 10 months ago 0 comments

Comment · 2 years ago

Sharing some interesting (preliminary) results for the 2024 State of Production ML Survey:

    Deploying ML: 36% take 1-3 months, 21% 3-6 months

    Experiment Tracking: 42% use MLFlow, 10% Spreadsheets

    Feature Stores: 53% use none, 28% Custom-built

    Vector DB: 55& use none, rest unconsolidated

    Training: 27& use custom-built, 21% Databricks

    Serving: 56% use custom-built (+ FastAPI/Flask)

    Monitoring: 50% use none, 24% Custom-built

    Diversity: Only 4% identify as female

This is a community initiative and the data will provide all of us in the community with actionable insights to improve the ecosystem. We aim for this input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community

We are opening this survey until the end of October, and we'll publish the results for the community to derive useful insights! If you can please take two minutes to share your experience: https://bit.ly/state-of-ml-2024

You can also check out the preliminary results here: https://ethical.institute/state-of-ml-2024 - we are building an interface for basic slice and dice to enable extracting further insights (but still early WIP so feedback appreciated). Final results / report will be published end of October!

The State of Prod ML: 36% take 1-3mo to prod, 10% use spreadsheets for tracking ethical.institute

3 axsaucedo 2 years ago 1 comments

Comment · 2 years ago

Kompute author here - thank you very much for sharing our work!

If you are interested to learn more, do join the community through our discord here: https://discord.gg/MaH5Jv5zwv

For some background, this project started after seeing various renowned machine learning frameworks like Pytorch and Tensorflow integrating Vulkan as a backend. The Vulkan SDK offers a great low level interface that enables for highly specialized optimizations - however it comes at a cost of highly verbose code which requires 800-2000 lines of code to even begin writing application code. This has resulted in each of these projects having to implement the same baseline to abstract the non-compute related features of the Vulkan SDK.

This large amount of non-standardised boiler-plate can result in limited knowledge transfer, higher chance of unique framework implementation bugs being introduced, etc. We are aiming to address this with Kompute. As of today, we are now part of the Linux Foundation, and slowly contributing to the cross-vendor GPGPU revolution.

Some of the key features / highlights of Kompute:

* C++ SDK with Flexible Python Package * BYOV: Bring-your-own-Vulkan design to play nice with existing Vulkan applications * Asynchronous & parallel processing support through GPU family queues * Explicit relationships for GPU and host memory ownership and memory management: https://kompute.cc/overview/memory-management.html * Robust codebase with 90% unit test code coverage: https://kompute.cc/codecov/ * Mobile enabled via Android NDK across several architectures

Relevant blog posts:

Machine Learning: https://towardsdatascience.com/machine-learning-and-data-pro...

Mobile development: https://towardsdatascience.com/gpu-accelerated-machine-learn...

Game development (we need to update to Godot4): https://towardsdatascience.com/supercharging-game-developmen...