My company is struggling with scaling up our applications and APIs because a good chunk of the core data lives within a single database. This DB/Server has been fined-tuned over the years to handle the ever-increasing load, but it's not sustainable. I am investigating moving some of this data into an event-stream as a path forward to reduce pressure on the DB and facilitate scaling APIs.
The high-level architecture:
- DB change detection publishes events to an immutable event stream.
- Event-stream processing service projects the data from the event-stream into a database for particular API.
- Data served from the API is sourced from the projected data in the API database.
The trade offs with this architecture are:
- Eventual consistency
- Copies of data
- Managing complexity
- Schema changes
I'm looking for some real-world examples of companies that have made the transition from a single DB to an event-driven architecture. Do you know of any case studies, interviews, etc. ?
The high-level architecture:
The trade offs with this architecture are: I'm looking for some real-world examples of companies that have made the transition from a single DB to an event-driven architecture. Do you know of any case studies, interviews, etc. ?