joshrivers
Joined 13 years ago
Joined 13 years ago
I remember the panic of the ‘90s about the coming disruption from internet technologies, but at best it has been a horizontal revolution, rather than a vertical one, and the same wealth and power centers have survived.
You would think that a new, superior set of technology and process would revolutionize, say, the hiring process. Instead, the brute force of lots of money provides a moat where HR, hiring managers, teams, and candidates are protected from meaningful disruption and progress.
All that said, my comment is almost certainly nonsense. I was referring to the broad sway of competitive advantage and the inertia of inadequate equalibria (more nonsense, surely). If you believe I am not properly building a warrant against the technical meanings on investment analysis terminology, you are right.
After all of that resolves, it's pretty hard to emotionally connect with the rest of the story, since you are exhausted from all that came before. The final act is worthwhile and interesting, it's just a pretty big change in direction.
What is depends on is how much horizontal scale you expect to have. At a small scale, you can use something like postgres as a centralized broker of sequence. A single postgres cluster can sequence a lot of records in a hurry, no problem.
At a larger scale you might need to decompose your domain into multiple, independently ordered aggregates. Now you have multiple choke point brokers, each of which can handle a lot of records, but which don't block each other.
Step it up another notch, you can use a consensus algorithm (paxos or raft) to get a cluster of machines to agree on the state of the sequence, and perform optimizations such as assigning blocks of sequence by shard and node.
I recommend not over building your event storage architecture until you measure your actual needs. There are cheap ways and huge expensive ways, and building too much is an easy way to make your project fail. Also, one of the nice things about a sequenced set of events is that it is pretty easy to replay into your new, faster event storage later once you've had the happy problem of too much success.
Right now the compliance world is addicted to firewalls, to the detriment to reasonable appsec. In my fantasy world, I'd like the auditors to be telling companies 'in 5 years, you won't be allowed to firewall your business network, and if you aren't secure without the crutches, then no certification for you.' That would light a fire under management to care about software quality all over the place.