isaac-harvey
Joined 1 month ago
Karma
6
Submissions
9
Member since
Oct 5
Recent submissions
Comment · 7 days ago
Comment · 23 days ago
Cool idea. How many are in the list already?
Comment · 23 days ago
In Australia there are really two property listing sites: Domain and RealEstate. This seems like something they might want to integrate.
Comment · 26 days ago
I feel like most consulting companies are hired because they are a brand that does a service and has some reputation. I think there will be a whole class of tech businesses that fill that niche. They might need a lot more human interaction than classic SaaS has been used to though. So I guess: build a niche and a reputation?
Comment · 1 month ago
This sort of incredibly simple, crisp idea is what changes the world. Great job!
Comment · 1 month ago
I mean, cracking anything below the nlogn bound implies that there might be much more room for improvement. Often a very minor win over the theory opens up enough extra attention to later truly move the needle.
Comment · 1 month ago
I really hope Germany will be able to find a good way out of their high carbon situation. Seems they desperately need to make sure the transition is cheap enough too, to keep industry competitive. Great data, thanks!
Comment · 1 month ago
Apart from the name, it's a decent idea. I just wonder how much it's adding for the newest models? I've found them to be better than any of my peers for data viz out of the box
I'm convinced DuckDB is the only engine needed to cover most real-world data pipelines. I've tested it on data scales up to terabytes, and running on appropriately-sized single machines, it's consistently beaten distributed engines like Spark. I think the age of distributed computing for most pipelines might be waning.
You can see the benchmark results for some tests here, where at 1 TB DuckDB 2.0 ran 10 times faster than (untuned) Spark: https://duckstring.com/blog/duckdb-pipeline-engine
Some specific features of Duckstring:
- Data transformations are treated like software packages, allowing the DAG to be implied by dependencies
- A pull-based orchestration system, throttling every process both upstream and downstream of the bottleneck so that nothing runs if it can't be consumed
- An incremental framework based on DBSP that handles updates and deletes efficiently
- CLI-first, so that you can point your own AI agents at the platform rather than relying on a janky integration
The link is to an intro story, which passes on to a Playground app to help get a feel for the orchestration method. Link if you want that directly: https://playground.duckstring.com
You can also just go straight to the site or the project on GitHub: https://duckstring.com https://github.com/duckstring-dev/duckstring
It's pre-1.0, though has been in active development since January, and is ready to try on real pipelines. I'd love to hear your feedback!