I think we need to caveat _what_ is a database when we make statements like this:
> Databases. They are the single source of truth for our most critical business data, yet as engineers we tend to overlook tooling with this in mind.
Most folks, like myself, will assume that we're talking about traditional concepts of databases, like RDBMS or Key/Value stores etc. But over time, after building distributed systems, it is important to realize that _logs_ are also a form of a streaming database and we need it to treat it like such.
This is the basic principle behind the venerable article by Jay Kreps (one of the authors of Kafka): The Log: What every software engineer should know about real-time data's unifying abstraction [1].
Once we start thinking this way, then we realize that everything ends up having a schema (protobuf, thrift, avro, etc.) -- just a "database" with a completely different view point.
IMHO, and consequently this is true where I work, The Log is considered the single source of truth and what is traditionally referred to as a "database" is just a "just-in-time" view of data that is materialized from The Log.
Anyway, that is just my 2c.
[1]: https://engineering.linkedin.com/distributed-systems/log-wha...)
> Databases. They are the single source of truth for our most critical business data, yet as engineers we tend to overlook tooling with this in mind.
Most folks, like myself, will assume that we're talking about traditional concepts of databases, like RDBMS or Key/Value stores etc. But over time, after building distributed systems, it is important to realize that _logs_ are also a form of a streaming database and we need it to treat it like such.
This is the basic principle behind the venerable article by Jay Kreps (one of the authors of Kafka): The Log: What every software engineer should know about real-time data's unifying abstraction [1].
Once we start thinking this way, then we realize that everything ends up having a schema (protobuf, thrift, avro, etc.) -- just a "database" with a completely different view point.
IMHO, and consequently this is true where I work, The Log is considered the single source of truth and what is traditionally referred to as a "database" is just a "just-in-time" view of data that is materialized from The Log.
Anyway, that is just my 2c.
[1]: https://engineering.linkedin.com/distributed-systems/log-wha...)