karakanb
Joined 9 years ago
Joined 9 years ago
This looks very interesting, letting people run these in any box they own. I very much agree with the sentiment that there are no proper tools that let you run any coding agent without being locked to a single provider. I just want to run opencode or pi somewhere in a box without having to spin up all of them in my machine, let alone being able to trigger them from within another prouduct as a background agent.
Would pi-pod allow me to standardize pi config on a team/project level so that other people in my org can also use them on the same private infra?
This is obviously a space we are very much interested in, so it is definitely nice to see approaches that attack the same problem. I haven't played with dbtcharts in depth yet but it looks very similar to DaC in principle, and also in the actual spec. I believe the industry definitely needs solutions like this to help get out of the legacy BI tools as Bİ is one of the biggest bottlenecks for AI adoption in large orgs.
Excited to see further competition in the space, nice launch!
I built it primarily for ourselves: we are building an AI data engineer, and we need a way to run many of them in parallel securely. An API for this seemed like the most obvious path forward. It makes it trivial to bring agentic capabilities into any product surface without having to deal with sandboxes, reliability issues, compatibility problems, and more.
I think it also makes sense from OpenAI's perspective to do this, but also we did find ourselves needing to change models and harnesses quite a bit, which is why I think this needs to be a layer of its own above the labs. It also needs to be a layer above the sandboxes, since many of them are quite brittle.
Overall, I expect a lot of the agent implementations to move in this direction. I think this is a lot saner for engineers to implement and maintain, and it makes it trivial to build agentic stuff into products.
It seems like a relatively straightforward marketing article. I was pleasantly surprised to learn about Erathos though, nice product!
I am personally not a big fan of CDC in prod. Streaming data movement is generally prone to confusion, and it feeds into bad data patterns like hard deletes without any audit logs, no timestamps on updates or deletes, etc. which are usually the reason why batch loads cannot be utilized. They require a decent operational understanding of the underlying database, and have some gotchas like the Erathos folks mentioned in the article. We offer CDC both in our cloud platform, as well our open-source tools, but if I could, I would always pick an incremental batch load with a cursor value over a CDC connection.
I understand it is sometimes required due to organizational complexity or legacy database reasons, mine is just a personal preference.
If anyone is looking for an open-source CDC tool that runs as a standalone Go CLI, check out ingestr: https://github.com/bruin-data/ingestr
For context: Bruin is a data platform that does ETL + BI. We do a lot of things open-source, and recently we launched Dashboard-as-Code: https://github.com/bruin-data/dac
The idea behind DaC was that using AI agents would require a more code-driven approach. That turned out to be directionally right, however it fell short in terms of unlocking even more value as people became more ambitious.
That's why we built data apps: - These are React apps that connect to your database / data warehouse - They work with all kinds of databases + external APIs - They can make external requests in a secure way without the credentials being embedded into the app itself.
This allows building all kinds of interesting and capable UIs with them. The dashboards you build become interactive internal apps that works with your data.
I have written a bit more about them here: https://getbruin.com/blog/announcing-data-apps
I would love to hear what you all think about generating more flexible dashboards on top of warehouses and what could be different. I wanted to offer this as a Show HN for folks to try out but it is currently a bit too expensive to run this publicly for us, that's why we are initially enabling access to the feature only to a select set of customers.
Let me know if you have any thoughts or what would you imagine seeing in this space.
About the staging dump, I haven't tried it but I don't see why it wouldn't work? I'll give it a try.
Yes, we'll create and share a helm chart, although we already publish Docker images so it should be trivial to spin up a chart if you wanted to.