tiwarinitish86

When everyone has access to roughly equivalent models, the differentiator is what you feed your model. It is time to look at data retention

Infrastructure has become a key driver of observability cost. More than data volumes, this is because of approaches like replication, cross AZ data transfer and almost no compression.

In this post we explain the infra cost of running Parseable for 100TB telemetry data on a daily basis. This is based on extrapolating ingestion volumes for an hour.

Looking forward to your comments / feedback on this!

Traces are one of most difficult observability signals to work with. You need a lot of work on the application and infrastructure instrumentation to get it right.

Once the hard work is done on the client side, you'd expect to extract max value from the this trace data (and not set small retention to save cost).

We worked hard to understand (and explain) how OTEL traces can be efficiently stored, queried and analysed on Parseable using the ubiquitous Parquet file format. Read on!

Columnar formats allow side stepping of some of the most critical issues that plague time series databases. Each label essentially becomes a column which is then efficient to parse through.