de107549
Joined 12 years ago
Karma
15
Submissions
20
Member since
Mar 14
Recent submissions
Comment · 1 year ago
Pretty crazy: "That’s 12 megatons of biomass—more than all the wild birds and mammals taken together."
Comment · 1 year ago
GitHub Actions power modern CI/CD workflows, but debugging failures and optimizing performance can be a challenge due to limited built-in observability. This post explores how to enhance GitHub Actions monitoring by integrating OpenTelemetry tracing. By instrumenting workflows, engineers gain deeper insights into execution times, dependencies, and bottlenecks. The guide walks through setting up OpenTelemetry in GitHub Actions, exporting traces, and visualizing them for better debugging and optimization. If you’re looking to improve your CI/CD observability and reduce blind spots, check it out!
Comment · 1 year ago
You’re absolutely right — while most observability vendors support receiving data via OTLP, they often convert it into a proprietary data format. This strips away the power of OpenTelemetry’s semantic conventions, such as OTel Resources. These conventions are incredibly powerful for adding context and correlating metrics, logs, and spans seamlessly. If you haven’t tried leveraging them yet, it’s worth it — the difference is huge.
Another key source of lock-in is dashboards and alerts. Teams often invest time creating hundreds of dashboards and alerting rules, only to face the painful process of recreating them when switching tools.
At Dash0, we tackle these lock-in points head-on: 1. PromQL for everything — not just metrics but also logs and spans. Plus, you can import/export Prometheus alerts (as code), making it easy to reuse existing projects like Awesome Prometheus Alerts: https://samber.github.io/awesome-prometheus-alerts/ 2. Perses for dashboards — a CNCF-backed open standard. You can import/export dashboards (as code), ensuring portability and independence from any vendor.
By adopting open standards like OpenTelemetry, PromQL, and Perses, you keep your observability stack flexible, portable, and future-proof — no more vendor lock-in.
Comment · 3 years ago
OTelBin will help developers adopt OpenTelemetry faster and more easily by providing a visual editor and validator for OTel Collector configurations. We are launching with built-in support for the OpenTelemetry community releases and the AWS Distro for OpenTelemetry, and we look forward to adding more!
OTelBin is our first product. It is free for everyone. The full code is available under the Apache 2.0 license on GitHub.
Our key features are:
- Visual Configuration & Pipelines: OTelBin provides an intuitive visual representation of your OpenTelemetry collector configuration. Say goodbye to trying to make sense of endless lines of YAML – see your data flow at a glance and make changes effortlessly and see your pipelines synched in real-time.
- Code Editor with Syntax Highlighting & Code Completion: OTelBin includes a built-in code editor with syntax highlighting, code completion, and basic validation (YAML validity, schema and basic collector principles like that you must declare the receivers you reference). Get the best of both worlds with code-based configuration and a visual representation of your configured pipelines.
- Validate against your Distribution: Depending on which components are included in a collector binary and the versions of its components, there are many version-specific aspects to fully validating a configuration. We have added support for OTelBin distributions (e.g., the official contrib and core builds) and their versions. OTelBin already supports both official builds of the OpenTelemetry collector and the AWS Distro for OpenTelemetry, and we would love to add more distributions with the help of their maintainers.
- Share your configuration easily: Collaborate effortlessly with your colleagues by generating shareable URLs for your configurations.
This is just the start of our journey, and we would love to get your feedback on OTelBin. Contact me or join our OTelBin Slack channel on the official CNCF Slack.
Comment · 10 years ago
Yes, I didn't compare to Wavefront as I have only basic insights and therefore cannot make a valid statement.
Competition depends on the uses case - if you are using a tool like SignalXF for custom metric analytics, then we are no competition as our focus is monitoring of applications and its underlying infrastructure.
We are an Application Performance Management (APM) solution and therefore compete more with tools like New Relic oder AppDynamics. Theses tools are sadly only used for troubleshooting in 90% of the cases and not for management or monitoring. They also do not work in highly dynamic and scaled environments as there "model" is too static. (which they try to fix with their analytics offerings)
This is what we want to change and were we add the whole stack to the game to analyze all the dependencies and help finding root causes quickly and monitor and predict the KPIs of your applications, services, clusters and components.
We integrate with solutions like SignalFX if needed but I have really good experience to do "dashboarding" with more business related tools like Tableau or QlikView - this also offers application owners an easier way to aggregate the monitoring data and metrics on a higher (business) level, where tools like Instana offer the instrumentation data as an input.
Comment · 10 years ago
There were different book that impacted different stages of my career:
As a programmer :
1. The C++ Programming Language by Bjarne Stroustrup.
2. Operating System Concepts by Silberschatz
3. Compilers by Aho
As an agile software developer:
1. eXtreme Programming by Kent Beck
2. Pragmatic Programmer by Hunt and Thomas
3. Continuous Delivery by Jez Humble
As an architect:
1. Domain Driven Design by Evans
2. Pattern-Oriented Software Architecture by Buschmann
3. Envisioning Information by Edward Tufte
As a CEO:
1. Good To Great by Collins
2. Lean Startup by Eric Ries
3. Beyond Budgeting by Pfläging
Comment · 10 years ago
I agree that streaming and timeseries queries/scans are two different approaches which can solve the problem in the same way. With instant vectors of Prometheus queries you can operate very similar to windows and if you do the right queries and take care that it works in-memory you also should get similar performance and throughput.
My point was more about the framework you get and how easy it is to apply analytics to streams/queries. SignalFx seems to have a nice workbench for this with direct visual feedback in the UI, so that you can work on existing data to get the right result.
As said we at Instana think that most people will not be able to build a sophisticated monitoring solution with these types of frameworks as they don't have the time to do it and maybe even not the analytical domain knowledge. You can see that SignalFx is adding specific knowledge for some technologies. I give you two simple examples to show that it is not easy:
- How would you predict if a file system is running out of disk space?
- How would you predict if you should add a node to a Cassandra cluster because it is running out of capacity (and it can take some serious time to add a node, so you should know in advance)?
Already the disk space problem is hard to solve - linear regression and basic algorithms will not work.
Now think of hundreds (or thousands) of services running on a dynamic container platform and new services released on a daily or even minute basis - with lots of different technologies involved...
No question that you can build a good monitoring solution with Prometheus, SignalFX, DataDog etc - but it will take a serious amount of time, consulting and dev teams involved adding the right instrumentation, metrics etc. And you need a lot of analytical knowledge. I can even imagine that there are situation were tools like Prometheus are a better choice - especially if you have a very strict set of technologies and communication framework and really good people to do a very specific set of "rules" for this environment.
We've added a domain model to our product (all the mentioned product have a generic metric model, but no semantics that describe servers, containers, processes, services and their communication which is the domain of system and application monitoring): Our Dynamic Graph.
And yes, we are using something very similar to Zipkin to get the dependencies between services. Here a are two blog entries describing the approach:
- About distributed tracing: https://www.instana.com/blog/evolution-tracing-application-p...
- How we safely instrument code: https://www.instana.com/blog/how-instana-safely-instruments-...
Mirko
Comment · 10 years ago
Let's first say that I am the co-founder an CEO of Instana, but I am trying to give a generic answer so that I don't "attack" competitors.
Most of the mentioned tools in this thread, including Datadog, SignalFX etc are using a simple agent to collect data - see Datadog agent on GitHub: https://github.com/DataDog/dd-agent or statsD (https://github.com/etsy/statsd) that is mostly recommended by SignalFX who have no own agent. Tools like Prometheus work similar.
On the backend side you can see two approaches for data store technology: A time series based approach like DataDog or Prometheus and a Streaming based approach like SignalFX - stream are the superior approach in my point of view as they allow for realtime approaches and stream (window based) analytics. There is a third category which is similar to time series but more "log" centric like the ELK stack or tool like Splunk.
On top of the data store these tools give you the ability to build your own dashboards (and provide standard dashboards for standard technology) and a alerting based on thresholds. They also allow to add you own metrics via API which can be used to add application specific data. They also give you a query API to query and combine the data in the store. So overall this is a Lambda architecture for monitoring data.
I would say that SignalFX is the most sophisticated one but the framework to work on stream is much more complicated then DataDogs time series approach so people go the easier way.
The problem with all of these tools is that they rely on the user to build dashboards, thresholds and in case of a problem do the correlation to find the root cause of the problem.
To correlate you need to understand the dependencies of the system components. As an easy example if service A has a performance issues because it calls service B that has a CPU problem, you need to know that A calls B and correlate the latency of A with the latency and CPU of B to find the root cause. You can discover/model dependencies with tools like Zipkin (https://github.com/openzipkin) or Spring Cloud Sleuth (https://cloud.spring.io/spring-cloud-sleuth/) which are based on the Google Dapper paper. You could even add or log the Span ID to the metrics/logs so that you can correlate them automatically.
Typically if you do so manually it is a disaster for change. All your correlations (and even dashboards) will not work if the topology of your services changes. Which is quite normal in the microservice world.
Instana uses a stream based approach similar to SignalFX BUT we combine this with a graph database that holds the dependencies of all physical and application dependencies. Our agent automatically discovers all the components and dependencies and adds them to the graph in realtime - including containers etc.
We then use the Google Four golden signals + Capacity (that was added by Netflix as the fifth one) to analyze the KPIs of the services and apply machine learning on it. That way we don't need manual thresholds which are also hard to maintain when things change a lot. If we see e.g. slow response times or sudden drops in requests or high error rates, then we analyze the dependency tree of that service to find the issues that are related to the problem and generate an incident for that - as we also discover changes, we add them to the incident as most often a change is the reason for a problem. I've written a blog entry on the Dynamic Graph: https://www.instana.com/blog/monitoring-microservice-applica...
Hope this answers you question.
Mirko