The open-source infra we use allows us to have a highly-available, scale-able system.
Kafka - buffers data ingest so that we never lose recordings.
Redis - acts as a cache, optimized for write-heavy processing in our app.
Postgres - stores metadata for the data we record.
ClickHouse - stores logs and other time-series data that need to be aggregated.
We're looking to simplify the system and replacing InfluxDB + OpenSearch in the near future with ClickHouse as a more performance and cost-effective solution that has proven simpler to maintain.
Kafka - buffers data ingest so that we never lose recordings. Redis - acts as a cache, optimized for write-heavy processing in our app. Postgres - stores metadata for the data we record. ClickHouse - stores logs and other time-series data that need to be aggregated.
We're looking to simplify the system and replacing InfluxDB + OpenSearch in the near future with ClickHouse as a more performance and cost-effective solution that has proven simpler to maintain.