Recent submissions

Comment · 9 days ago
Same from AU
Comment · 2 months ago
Comment · 4 months ago
Engineering hasn't gone away, you're now just directing things at a higher level. You are now a architect & manager (but you're managing agents not people).

Who sometimes has to deep dive & mentor a agent on solving the right problem.

Comment · 9 months ago
Tell us why you think so at least.
Comment · 1 year ago
I'm not sure I would want this even if I could have it TBH. Engingeering org size is about ~200 with infra/sre/ops around ~25.

Different teams want to move at difference cadences. At a certain scale splitting up things feels a little more natural (maybe I am stockholmed by prior limitations with TF though or just used to this way of operating now).

But even then, we're moving to k8s operators to orchestrate a bunch of things and moving off terraform apart from the stuff that doesn't change much (which will eventually get retired as well). Something like https://www.youtube.com/watch?v=q_-wnp9wRX0

Terraform variable management is our larger problem (now/nearterm) when we have to deploy numerous cells of infra that use the same project/TF files with different variables. Given the number of projects/layers of TF getting cell specific variables injected is meh.

Those variables are instance size, volume size, addresses, IAM policy, keys etc.

This is in the b2b saas world with over a million MAU. We've got islands of infra for data soverignty, some global cells where each cell can communicate back / host some shared services (internal data analytics, orchestration tooling, internal management tooling and the like).

Comment · 1 year ago
This is awesome, I will be trying this out in the coming months. Its just made it to the top of my R&D shortlist for things that could massively simplify our data stack for a b2b saas.
Comment · 1 year ago
https://www.youtube.com/watch?v=oYiFTBO67uU

https://innovation.ebayinc.com/stories/graphload-a-framework...

It looks like people are using it to build graph related models on it.

I am looking at it & considering doing something similar for graph data sets. As well as a transactionally safe key value store to store roaring bitmaps.

Comment · 1 year ago
Caught a steam train that operates on the mainline in NSW, Australia last year.

https://www.picnictrain.com.au/

We had to stop & wait at various points on our return day trip for other passenger services to pass us. Had to wait for some freight services to clear a block at a few places too.

There was an auxiliary diesel unit not actively used apart from for power AFAIK (and additional emergency mobility should things go kaboom/excess wheel slip).

Maybe that has some the extra safety systems on it though.

Comment · 1 year ago
I started looking into podman quadlets on the weekend. All because docker & nft (nftables firewall) don't play nice together.

I have my finely crafted nftables ruleset that I want to keep. Docker is going to abuse the crap out of my ruleset/make me jump through extra hoops to have it work.

Having the containers operate under systemd looks fine (ignoring the fact that it is systemd, I guess I have finally accepted systemd is here to stay).

I was interested in using traefik & docker labelling to have ingress sorted out and I believe this will all play nicely enough to replace docker-compose on a single server.

Comment · 1 year ago
Unifying things slows engineers down, so probably not (for some time).
Comment · 1 year ago
https://youtu.be/qcNNB2qZ5kE?t=2776 around this time code. If the USA doesn't want a nation to fight then there are more issues than a F35 software kill switch etc.
Comment · 1 year ago
This looks super interesting.

Just starting to review it but my front of mind questions: 1) How do I handle persistence? Looks like some code is missing. 2) Do you support multi-tenancy (b2b saas graph backend for handling relations scoped to a tenant)

Thanks

Comment · 1 year ago
Interesting in hearing some thoughts about using roaring bitmaps stored in a bytea postgres column to represent adjacency matrixes.

I was thinking that given RDS has support for plrust and PostgreSQL's SPI I could use the fact they support croaring-rs there as a crate and build upon that.

I figure I can use that to represent many graph's with say 100s to ~100m nodes and many relations between these things. But each graph would be tenanted to a tenant (company/b2b saas use case).

I was thinking that by using plrust storing the roaring bitmap on the DB server in a bytea and using SPI, I can benefit from the minimal network overhead to mutate and query against the bitmap with croaring. Using SPI locally in the DB server I eliminate network overhead shipping that back to my application code.

PostgreSQL also gives me transaction safety to updates etc. And a bunch of support for other column base data such as my tenant ID column, some JSONB for relationship metadata to query on etc.

Basically something like https://jazco.dev/2024/04/20/roaring-bitmaps/ but on postgres. Given I need to support many tenanted graphs & we're already using citus this seems like something that is feasible at a larger scale too.

I was wondering though if I am going to need to create some operator classes to allow me to index relations a bit better (probably seems likely I think).

I am aware of https://github.com/ChenHuajun/pg_roaringbitmap but would prefer to use int64s and maybe start out on RDS instead of having to add another workload to our self hosted citus cluster/s.

Happy to be told I am fool and any insights would be nice. I am potentially going to try this out on some of our data sets we have because our product team is basically laying out a vision where they want us to have a graph powering a bunch of things.

I don't like the idea of neo4j when we're already deep into PostgreSQL for a bunch of workloads (~20+ TB table workloads etc so we have some reasonable inhouse PG experience).

Also huge thanks to the author of the blog post. I had been looking at pgRouting and wondering with a tilted head.. hmm seems like we can just use this as a graph DB. So that is also on my list to test out.

Comment · 2 years ago
Where does one find out about all of these conventions for rails?
Comment · 2 years ago
250 Pitt St Sydney around 2002-2003.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton