Data observability is a relatively new discipline in the fields of data engineering and data management. While many are familiar with the longstanding concepts of observability and monitoring in enterprise IT networks and infrastructure, data observability has only really come into the spotlight in the last two years. However, it has managed to turn a lot of heads in that short time.

The internet of things is one of my favorite topics. IOT enables low-powered connected devices that opens gateways from the digital to the real world. While I love tinkering away with an Arduino sketch and the latest Espressif or Arduino board, there is always an air of frustration when trying to build out what at first seems like simple functionality using one of these “smart devices” because of the limited view we have into their operations.

It may surprise you to hear, but Honeycomb doesn’t currently have a platform team. We have a platform org, and my title is Director of Platform Engineering. We have engineers doing platform work. And, we even have an SRE team and a core services team. But a platform team? Nope. I’ve been thinking about what it might mean to build a platform team up from scratch—a situation some of you may also be in—and it led me to asking crucial questions. What should such a team own?

For several decades, IT monitoring has been deployed in different forms. The focus of IT monitoring has been to gather metrics about the operations of an IT infrastructure’s hardware and software assets to ensure that all the key functions are being performed as expected to support applications and IT services. In the recent past, the term Observability has been used as a synonym for “modern monitoring”.

Written by Liz Fong-Jones and Phillip Carter. OpenTelemetry, also known as OTel, is a CNCF open standard that enables distributed tracing and metrics collection from your applications. At Honeycomb, we believe that OpenTelemetry is the best way to ingest the high-cardinality and high-dimensional data that every system, no matter how complex or distributed, needs for observability.

Among the 12 greatest stressors in life, six revolve around healthcare issues. From loss of a loved one to pregnancy and even retirement, these events often involve interactions with healthcare services — interactions that can either add to an individual’s stress or, ideally, help alleviate it.

Organizations of all sizes rely on their observability data to drive critical business decisions. Production Engineers across Development, ITOps, and Security use it to understand their systems better, respond to issues faster, and ultimately provide more performant and secure user experiences. But while the value of observability data is well understood, teams struggle to derive value from it.

As long as humans have written software, we’ve needed to understand why our expectations (the logic we thought we wrote) don’t match reality (the logic being executed). To that end, we developed techniques to help measure reality—logging text strings, or capturing aggregated metrics—and persevered, seeking out newer and fancier logging or monitoring solutions over the intervening decades.

If you’re finding it a little difficult to sift through all the varying, often conflicting information surrounding the topic of network observability, you’re certainly not alone.

Today, we're announcing the launch of Honeycomb Service Map. This isn't your grandparent's version of a service map. This feature reimagines what it is that you want to know or investigate when looking at visualizations of how your services communicate with one another.

Understanding if your applications are not just available but also functioning as expected is critical for any organization. Third-party dependencies and different end-user device types means that infrastructure monitoring and application observability alone are not enough to spot and minimize the impact of application anomalies.

We’re excited to announce our beta launch of Kentik Kube, an industry-first solution that reveals how K8s traffic routes through an organization’s data center, cloud, and the internet. With this launch, Kentik can observe the entire network — on prem, in the cloud, on physical hardware or virtual machines, and anywhere in between.

The spine is essential to perform every activity, like crawling, walking, or swimming. Just as the spine is necessary to enable these functions, your Kubernetes infrastructure needs a backbone to be efficient and effective. So if Kubernetes clusters act as the spine of your architecture, then Kubernetes nodes are like the vertebrae — they make up a Kubernetes cluster in the same way the vertebrae form the spinal column.

If you build or maintain code in GitHub, the Honeycomb Buildevents Action can help you optimize the performance of your build pipelines in GitHub Actions. This blog introduces you to the gha-buildevents Action and a new hands-on quickstart guide that will show you the inner workings of GitHub Actions workflows, the buildevents tool, and the Honeycomb UI.

Ask any cloud software team using Kubernetes (and most do); this powerful container orchestration technology is transformative, yet often truly challenging. There’s no question that Kubernetes has become the de-facto infrastructure for nearly any organization these days seeking to achieve business agility, developer autonomy and an internal structure that supports both the scale and simplicity required to maintain a full CI/CD and DevOps approach.

This article was originally published on HackMD and is reposted here with permission. Presently organizations are unable to monitor millions of embedded Linux devices in real-time. With so many different architectures and device types, aggregating telemetry and metrics and viewing that data in a centralized analysis tool is problematic. Onboarding embedded Linux devices into a telemetry service so that metrics can be easily observed is a significant challenge.

Today, Grafana Labs announced a strategic partnership with Isovalent, the creators of Cilium, to make it easy for platform and application teams to gain deep insights into the connectivity, security, and performance of the applications running on Kubernetes by leveraging the Grafana open source observability stack.

Getting existing telemetry into Honeycomb just got easier! With the release of the Datadog APM Receiver, you can send your Datadog traces to the OpenTelemetry Collector, and from there, to any OpenTelemetry-compatible endpoint. Often, evaluating a new tracing solution requires re-instrumenting your applications from the ground up in a new vendor’s tooling. It’s a pretty high bar to clear just to see if a solution is worth adopting.

Network topology can get very complicated in the cloud, especially when you’re sending data to external SaaS providers. You will likely need to configure gateways and firewalls and keep close tabs on those points of egress. However, if your infrastructure exists within AWS, there’s a much simpler way and that’s through an AWS PrivateLink endpoint.

When you want to direct your observability data in a uniform fashion, you want to run an OpenTelemetry collector. If you have a Kubernetes cluster handy, that’s a useful place to run it. Helm is a quick way to get it running in Kubernetes; it encapsulates all the YAML object definitions that you need. OpenTelemetry publishes a Helm chart for the collector. When you install the OpenTelemetry collector with Helm, you’ll give it some configuration.

Today, I’d like to share with you a new community-contributed integration that helps you optimize and debug your Gradle builds. This new Gradle plugin is available today, is free to use, and you can use it immediately with a free Honeycomb account.

Open source monitoring and observability tools can be found in production all over the world – whether they’re being used by startups or entire enterprise development teams. DevOps, ITOps, and other technical teams rely on tools like Prometheus, Grafana, OpenSearch, OpenTelemetry, Jaeger, Nagios, Zabbix, Graphite, InfluxDB, and others to monitor and troubleshoot their cloud environment.

One of my initial surprises upon joining Catchpoint about five months ago was to do with how much confusion there is in the observability market. Every single vendor has almost the same message around ensuring a great digital experience for your customers or employees or both. Of course, these experiences are critical to get right, but for the most part many of these solutions, at best, help to ensure that sites are live and available, and that they are reachable by some users.

In my work as a technical evangelist at Cribl, I regularly talk to companies seeing annual data growth of 45%, which is unsustainable given current data practices. How do you cost effectively manage this flood of data while generating business value from critical data assets?

Observability is a measure of how well we are able to infer the internal state of our application from its external outputs. It’s an important measure because it indirectly tells us how well we’d be able to troubleshoot problems that will inevitably arise in production. It’s been one of the hottest buzzwords in the cloud space for the last 5 years and the marketplace is swamped with observability vendors. Different tools employ different methodologies for collecting data.

Logz.io is one of Logz.io’s biggest customers. To handle the scale our customers demand, we must operate a high scale 24-7 environment with attention to performance and security. To accomplish this, we ingest large volumes of data into our service. As we continue to add new features and build out our new machine learning capabilities, we’ve incorporated new services and capabilities.

Earlier this month I hosted the “Everything You’ve Heard About Observability is Wrong (Almost)” webinar– thanks to all of you who attended. I wanted to follow-up with the attendees as well as those who were not able to join. As promised, it wasn’t the same old Observability presentation that we have grown accustomed to you know, all marketing with little value.

BubbleUp is Honeycomb’s machine-assisted debugging feature and is one of our most powerful differentiators. It leverages machine analysis to cycle through all of the attributes found in billions of rows of telemetry to surface what is in common with problematic data compared to baseline data. This explains the context of anomalous code behavior by surfacing exactly what changed when you don’t know which attributes to examine or index, dramatically accelerating the debugging process.

Observability is the ability to see and understand the internal state of a system from its external outputs. Logs, Metrics, and Traces, collectively called observability data, are three external outputs widely considered to be three pillars of observability. Now more than ever, organizations of all sizes must employ the necessary processes and technologies to harness the power of their data and make it more actionable.

Just a few short months ago, we talked about a bunch of updates to Honeycomb’s support for OpenTelemetry. To the surprise of no one, we’ve got more updates to share!

You know that old adage about not seeing the forest for the trees? In our Authors’ Cut series, we’ve been looking at the trees that make up the observability forest—among them, CI/CD pipelines, Service Level Objectives, and the Core Analysis Loop. Today, I'd like to step back and take a look at how observability fits into the broader technical and cultural shifts in technology: cloud-native, DevOps, and SRE.

StackState’s new 4T Monitors introduce the ability to monitor IT topology as it changes over time. Now your observability processes can trigger alerts on changes in topology that don’t match an ideal state, on deviations in metrics and events and on complex combinations of parameters. Monitoring topology as part of your observability efforts enriches the concept of environment health by adding the dimension of topology.

I am thrilled to share with you that Honeycomb now has a Field CTO: our very own Liz Fong-Jones. When Liz joined us, nearly four years ago, she was our first developer advocate and principal engineer. It was a bit of a risky move—on both our parts.

With an exploding volume of data and systems comes the need for observability, or the ability to understand the internal states of a system from knowledge of its external outputs. As a result, observability data's importance is at an all-time high. Businesses spanning every industry use it in various ways to respond to issues, increase agility, mitigate risk, and ultimately provide better experiences for their users. It’s an incredibly valuable commodity.

At Honeycomb, the datastore and query systems that we manage are sociotechnical in nature, meaning the move to observability requires a sociological shift as much as it does a technical one. We've covered the technical part in several prior discussions for our Authors’ Cut series, but the social aspect is a little squishier. Namely: How do you solve the people and culture problems that are necessary in making the shift to adopt observability practices?

To many engineers, the idea that you can accurately and comprehensively track your application's user experience using just a few simple metrics might sound far-fetched. Believe it or not, there are four metrics that aim to do just that. They're called the four Golden Signals and should be a core part of your observability and reliability practices.

Today, I’d like to tell you about a new community-contributed integration that connects Honeycomb to your ServiceNow workflows. My new integration reimagines what’s possible when connecting observability tools with ITSM systems. This post explains how it works and how to get started with it.

Observability is a methodology that you incorporate into your enterprise architecture to provide greater visibility into what is happening. It helps us determine the states of the system from their external outputs and allows technicians to identify bottlenecks, predict issues and mitigate them. As the architectures of IT systems are becoming more complex and distributed we use observability to meet the need to measure their internal states.

Observability is a hot topic in the IT world these days. It is oftentimes discussed through the lens of the “three pillars of observability”: Logs, Metrics and Traces. Indeed these telemetry signal types help us understand what happened, where it happened and why it happened in our system.

We all know that observability is a must-have for operating systems in production. But we often neglect our own backyard — our software release process. We noticed we made that mistake here at Logz.io. We were wasting time and energy in handling failures in the CI/CD pipeline, and made our Developer-on-Duty (DoD) shifts tedious. That’s why it’s critical to incorporate your observability practices into your CI/CD pipeline.

Building modern, cloud-native applications introduces new challenges to teams and organizations. As these systems grow and scale, struggles abound: inconsistent performance monitoring experiences across siloed tools, wasteful performance management practices with duplicated efforts, and mounting frustration from colleagues and customers. Surmounting these challenges requires multiple sources of data and truly unified observability.

Debugging is an unavoidable part of software development, especially in production. You can often find yourself in “debugging hell,” where an enormous amount of debugging consumes all your time and keeps the project from progressing. According to a report by the University of Cambridge, programmers spend almost 50% of their time debugging. So how can we make production debugging more effective and less time-consuming?

So, what is Universal Profiling™? Universal Profiling™ is fast emerging as an important component of observability. A standard feature inside hyperscalers since approximately 2010, the technology is slowly percolating into the wider industry. Universal Profiling™ allows you to see what your code is doing all the time, in production across a wide range of languages and can profile both user-space and kernel-space code.

Developing an observability practice brings opportunity for AppDynamics partners to expand customers' relevance and profit at every step of their digital journey.

From its inception as a powerhouse for logging, Elastic Observability has grown into a comprehensive solution for full-stack multi and hybrid-cloud observability. Given the increasing complexity of the cloud-native world, the major challenge for observability is twofold: getting deeper and more frictionless visibility at all levels of applications, services, and infrastructure, and making sense of the overwhelming amount of data that is available.

Another month has come to a close, so I’m back again to take you through what’s new and noteworthy from the month of September. If you missed last month’s blog, this will be a monthly recurring series to keep you posted with the latest and greatest at Honeycomb. There’s a ton to cover, so I’ll dispense with the preamble and dive right in.