Hi HN,
I'm looking for advice and insight on what y'all might use for an internally hosted logging solution in Kubernetes. Currently we use a self-hosted Graylog setup, but are finding it difficult to maintain as our system grows.
Here's our current setup:
- Multiple clusters
- Logs aggregated to a single Graylog setup, itself running in Kubernetes
- Logs are sent to Graylog via Fluentbit
Some problems we've had are:
- Index management in Graylog's ElasticSearch cluster is a PITA when you have many differently shaped log forms going to shared indices (managing separate indices per source is also a pain)
- Management of MongoDB in Kubernetes is frustrating and has been a reliability challenge
I'd love for us to be able to use a hosted logging solution but $$$ obviously. I'm aware of many other alternatives, but one of the things I've painfully learned is that a basic feature matrix only tells a very small piece of any story. The real education comes from running this type of tech and living with it through scale and various lifecycle events.
Some questions I have:
- What logging solutions are you using in your Kubernetes environment, and how has your experience been?
- How do you handle log retention and storage costs?
TIA
I'm looking for advice and insight on what y'all might use for an internally hosted logging solution in Kubernetes. Currently we use a self-hosted Graylog setup, but are finding it difficult to maintain as our system grows.
Here's our current setup:
Some problems we've had are: I'd love for us to be able to use a hosted logging solution but $$$ obviously. I'm aware of many other alternatives, but one of the things I've painfully learned is that a basic feature matrix only tells a very small piece of any story. The real education comes from running this type of tech and living with it through scale and various lifecycle events.Some questions I have:
TIA