supercoco9
Joined 4 years ago
Karma
14
Submissions
10
Member since
Mar 24
Recent submissions
Comment · 4 months ago
Or a bicycle!
Comment · 1 year ago
I don't think it would. The room should be called "Cloud Native Databases", as it was called last year. This is the CFP they issued on the FOSDEM mailing list
---
We're excited to announce the Databases devroom at FOSDEM 2026. The purpose of this devroom is to discuss developing databases that are designed to take advantage of the cloud native architecture in order to meet the demands of modern applications.
Instead of focusing on a single technology or community, our goal is to bring open source database communities and developers together to share their experience, learn from their work, and foster collaboration on challenges ahead.
Suggested Topics - Evolution of new database architecture for cloud native environments (e.g., to address scalability, elasticity, reliability) - Integrating AI/ML workloads with distributed databases - How to ease/streamline database migration (especially from legacy databases) - Achieving database upgrades without downtime - How to benchmark and tune distributed databases - Query optimization for cloud native databases - Tools/methodologies for monitoring, observability, optimizations, testing, etc. - Working with streaming data - Stories from end-users (both successes and failures) - Security and compliance for cloud native databases - State of open-source licenses and database communities - Emerging trends/use cases
These are just suggestions and we encourage you to be creative with your proposals! You can also view accepted talks from the 2025 devroom at https://archive.fosdem.org/2025/schedule/track/databases/.
---
I don't think the traditional talks that were scheduled at the PostgreSQL or MySQL and Friends DevRooms would be welcome there.
Comment · 1 year ago
QuestDB. It is built for finance workloads (can be also used for other timeseries data, like energy, or aerospace, but has heavy optimizations for common finance data patterns), it is very performant, it has been in used for years at large finance entities, and it is Apache 2.0.
Full disclosure: I am a Developer Advocate at QuestDB.
The Open Source edition does not limit any commercial use or the size of the machine you can install, as per the Apache 2.0 license terms.
If you want more enterprise-y options, like single sign on, or RBAC, there is an Enterprise edition. But Open Source is as performant as the Enterprise version. Enterprise offers also things like replication and TLS on all endpoints, which can be somehow replicated in Open Source with manual sharding or proxies.
Comment · 1 year ago
Hi! I am the author of that post :)
Exactly my thinking. It is alpha, and I am sure it will be vastly improved, yet...
Comment · 2 years ago
By the way, I presented this repository at the FOSDEM conference just a few days ago. If you want to have more context about what it does, or see a walk-though of the different components, the video is available at https://fosdem.org/2024/schedule/event/fosdem-2024-2871-inge...
Comment · 2 years ago
Thank you! (original article writer here)
DuckDB is awesome. A couple of comments here. First of all, this is totally my fault, as I didn't explain it properly.
I am trying to simulate performance for streaming ingestion, which is the typical use case for QuestDB, Clickhouse, and Timescale. The three of them can do batch processing as well, but they shine when data is coming at high throughput in real time. So, while the data is presented on CSV (for compatibility reasons), I am reading line after line, and sending the data from a python script using the streaming-ingestion API exposed by those databases.
I guess the equivalent in DuckDB would be writing data via INSERTS in batches of 10K records, which I am sure would still be very performant!
The three databases on the article have more efficient methods for ingesting batch data (in QuestDB's case, the COPY keyword, or even importing directly from a Pandas dataframe using our Python client, would be faster than ingesting streaming data). I know Clickhouse and Timescale can also efficiently ingest CSV data way faster than sending streaming inserts.
But that's not the typical use case, or the point of the article, as removing duplicates on batch is way easier than on streaming. I should have made that clearer and will probably update it, so thank you for your feedback.
Other than that, I ran the batch experiment you mention on the box I used from the article (had already done it in the past actually) and the performance I am getting is 37 seconds, which is slower than your numbers. The reason here is that we are using a cloud instance using an EBS drive, and those are slower than a local SSD on your laptop.
You can use local drives on AWS and other cloud providers, but those are way more expensive and have no persistence or snapshots, so not ideal for a database (we use them sometimes for large one-off imports to store the original CSV and speeding up reads while writing to the EBS drive).
Actually one of the claims in DuckDB's entourage is that "big data is dead". With the power in a developer's laptop today you can do things faster than with cloud at unprecedented scale. DuckDB is designed to run locally on a data scientist machine, rather than on a remote server (of course you have Motherduck if you want to do remote, but you are now adding latency and a proprietary layer on top).
Once again thank you for your feedback!
Comment · 2 years ago
Hi. Sorry if my query offended you.
I basically executed literally what Clickhouse recommends at their guides for deduplication https://clickhouse.com/docs/en/guides/developer/deduplicatio....
Of course you can also materialize with aggregations or just use a group by, or even force optimize of the table. But my point is that you don't really get exactly once guarantees. Whoever is querying that table needs to be aware than a `SELECT * FROM tb` might contain duplicates and needs to adapt their queries accordingly.
Comment · 3 years ago
I enjoyed reading about how technical decisions are made to make databases smarter and optimise without the need to relay on users fine-tuning obscure configuration parameters
I love data, big and small, and I enjoy processing it with streaming engines, ML models, SQL, NoSQL, graph, in-memory, and time-series databases. I prefer distributed, scalable, always-on systems.
Before working at QuestDB I spent over 20 years developing software professionally and sharing what I learnt with the community, including three years as a Developer Advocate at AWS and five years as a Google Developer Expert in Cloud. I've spoken at events in over 25 countries, mentored dozens of start-ups, taught for 6 years at different universities, and trained hundreds of professionals on cloud and data engineering.
For full bio and details, check https://javier-ramirez.com