gpaulbagetti
Joined 2 months ago
Karma
13
Submissions
13
Member since
Aug 25
erathos.com data ingestion for the AI era
View on Hacker NewsRecent submissions
Comment · 1 month ago
Wrote this after watching a client's query costs balloon from table layout decisions made early and never revisited. Mostly about partitioning/clustering tradeoffs people skip when moving fast.
Comment · 1 month ago
Follow-up to a piece we did on data org for cost, this one's narrower: which BigQuery cost levers actually move the needle vs. which ones are folklor
Comment · 1 month ago
hey thank you for your feedback! The description was giving the wrong idea indeed. I've just updated the article :)
Comment · 2 months ago
Hey! Glad to hear about Bruin, competition always pushes us to make our product better for our customers :)
Frankly, I am not a big fan of streaming either. But CDC and streaming are two different things. Streaming database replication is generally built on CDC, but you can use CDC with micro-batches too (which is what we do). I wrote about this in another article: https://www.erathos.com/en/blog/cursor-based-sync-vs-change-...
Imo, the key reason for offering CDC is that most databases aren't designed around "how can we extract data efficiently", therefore they often lack a reliable "updated_at" column for us to use as a cursor.
Comment · 2 months ago
Wrote this after seeing the same failure mode too many times: a MySQL sync job that looks fine in the dashboard but has been silently dropping deletes and intermediate updates for months, because it's comparing snapshots instead of reading the binlog. Tried to lay out exactly what has to be true on the MySQL side (row image, binlog_row_value_options, server-id, retention) for CDC to actually be complete, not just "eventually consistent." Happy to go deeper on any of it in the comments.