Observability pipelines solve some critical problems IT is facing today: the cloud environment has generated an unprecedented amount of data in recent years; enterprises now have multiple SaaS/cloud-based applications running; it’s becoming tough to know which of this massive volume of data needs to be processed for analysis vs. stored (often for regulatory reasons) cheaply; and dealing with growing numbers of source data makes the meaningful management of the problem only harder.

Data visualizations take complex information and present it in a clean and easy-to-understand visual. Done right, they can allow quick insight through easy pattern and outlier recognition. Done wrong, it can confuse, obfuscate, and lead to wrong conclusions. Yikes! Over the past few months, we've been hard at work modernizing Honeycomb’s data visualizations to address consistency issues, confusing displays, access to settings, and to improve their overall look and feel.

Artificial intelligence for IT Operations (or AIOps) has been playing an expanding role in helping SREs, DevOps, and developers effectively navigate the challenges around application and infrastructure complexity, pace of change, and data volume that characterize the operations landscape.

In today’s fast paced and constantly evolving digital landscape, observability has become a critical component of effective software development. Companies are relying more on and using machine and telemetry data to fix customer problems, refine software and applications, and enhance security. However, while more data has empowered teams with more insights, the value derived from that data isn’t keeping pace with this growth. So how can these teams derive more value from telemetry data?

The software industry has seen many evolutions. There is a new disruption in the market every five years or so. Software testing cannot remain isolated from all the latest trends and technologies. Testing strategies need to keep up with agile development, faster deployments and increasing customer demand for reliability and user-friendly interfacing. They need to be able to grow just as quickly and just as reliably as the business logic.

How is your organization handling Kubernetes observability? What tools are you using to monitor Kubernetes? Is it a time-consuming, manual process to collect, store and visualize your logging, metrics and tracing data? And, what are you actually getting out of all that investment? At Logz.io we’re trying to make this process easier for customers who are serious about Kubernetes observability. We’ve made significant investments in this area for Kubernetes use cases.

In today's digital age, organizations increasingly depend on their technology infrastructure to keep their operations running smoothly. These infrastructures include servers, networking equipment, IoT devices, and applications. The data generated by all this infrastructure (logs, metrics, traces) is known as telemetry data, which has a tremendous potential value to organizations. However, it can be challenging to control telemetry data and utilize it effectively.

Observability has become one of the most important areas of your application and infrastructure landscape, and the market has an abundance of tools available that seem to do what you need. In reality, however, most products - especially leading open source tools - were created to solve a single problem extremely well, and have added additional supporting functionality to become a more robust solution; but the non-core functionality is rarely best of breed. Examples of these are Prometheus and Grafana.

In data management, numerous roles rely on and regularly use telemetry data. The developer is one of these roles. Developers are the creative masterminds behind the software applications and systems we use and enjoy today. From conception to finished product, they map out, build, test, and maintain software.

Incidents happen. What matters is how they’re handled. Most organizations have a strategy in place that starts with log searches—and logs/log searching are great, but log searching is also incredibly time consuming. Today, the goal is to get safer software out the door faster, and that means issues need to be discovered and resolved in the most efficient way possible.

StackState commissioned Techstrong Research, a strategy and technology analyst firm, to delve into the current state of observability. The resulting report, “Observability Innovation Report 2023,” provides insightful information. 543 IT professionals were surveyed, globally, across 20 industries. The largest concentration of respondents were in the telecommunications, technology, Internet and electronics sectors, followed by financial services.

Observability has become a bit of a buzzword in the industry for the last few years. Exactly what "observability" means depends on who you ask, but most people would agree its about both: There's plenty of content out there telling you how to implement observability, or what good looks like. But what about bad observability? What are some anti-patterns to watch out for?

Observability data provides the insights engineers need to make sense of increasingly complex cloud environments so they can improve the health, performance, and user experience of their systems. These insights can quickly answer business-critical questions like, “what is causing this latency in my front end?” Or, “why is my checkout service returning errors?” Observability is about accessing the right information at the right time to quickly answer these kinds of questions.

Moving to the cloud is hard. Moving to the cloud and keeping systems secure, data governed, compliances met, and cyberattacks at bay, makes everyone’s jobs significantly harder. The number one concern we hear from Cribl customers about the cloud is, you guessed it — security. If you’re in this boat — eager to adopt the cloud ASAP but also worried about the risks that come with having sensitive data in the cloud — don’t fret. We’re here to help.

Elastic Observability provides a full-stack observability solution, by supporting metrics, traces, and logs for applications and infrastructure. In a previous blog, I showed you how to monitor your AWS infrastructure running a three-tier application. Specifically we reviewed metrics ingest and analysis on Elastic Observability for EC2, VPC, ELB, and RDS.

Gain executive-level insights on Cisco AppDynamics’ for on-premises, hybrid and cloud customers from Ronak Desai, Cisco SVP & GM AppDynamics and full-stack observability.

Most SaaS products have nice, organic growth when they work well. Employees log in, they click around and make stuff, then they share links with others who do the same. After a few weeks or months, there are thousand of objects. Some are abandoned, and some are mission-critical. Different people also bring different perspectives, so they name things that are relevant to their role and position in the team, which may be confusing to others outside their realm.

Not long ago, we announced the launch of Honeycomb’s Service Map, a new feature that gives users the ability to get an overall, filterable view of their system and how everything is connected, along with some exciting new enhancements to BubbleUp. What’s the story behind these changes? They make it even easier for developers to zero-in on issues, even when they are hidden in billions of lines of code.

Baking a delicious pizza in a wood-fired oven requires a combination of skill, experience and the right tools. The same is true for achieving optimal observability in a Kubernetes environment. In this post, we'll explore some of the lessons learned from baking pizza in a wood-fired oven and apply them to the world of Kubernetes observability.

Observability, monitoring, and telemetry are crucial for maintaining the performance and reliability of modern systems. Their concepts are often used interchangeably, but they have distinct differences that are important to understand. In this blog, we’ll explore each concept in detail, including key characteristics and examples of tools. We’ll also compare observability vs monitoring vs telemetry and discuss when it’s appropriate to use each.

If you were asked to evaluate how good crews were at fighting forest fires, what metric would you use? Would you consider it a regression on your firefighters’ part if you had more fires this year than the last? Would the size and impact of a forest fire be a measure of their success? Would you look for the cause—such as a person lighting it, an environmental factor, etc—and act on it? Chances are that yes, that’s what you’d do.

Observability has become increasingly important for IT professionals as the complexity of modern systems has grown. In the past, IT environments were typically composed of a few servers and applications that were all running on-site. However, with the rise of cloud computing, IT has become more distributed, with applications and services running on a wide variety of infrastructure and platforms.

Data routing is a crucial but complex task for companies of all sizes. Ensuring that the right data is sent to the right tools can be a time-consuming and difficult process, and when things go wrong, it can have costly consequences. This is why having a robust data routing strategy is essential for any organization.

Whether you’ve been following along with our Authors’ Cut series or doing some self-paced learning, our O’Reilly book Observability Engineering is one of the best resources for jumpstarting your observability journey. It serves as a blueprint to help you understand and map out the technical and cultural requirements of implementing observability into your organization.

By incorporating observability into your stack, you can better understand how your complex infrastructure operates, reduce downtime, and empower developers to quickly identify and fix problems. However, it now takes considerably more work, time, and money to build up observability for your infrastructure and applications. Over half of the firms polled employ eight or more observability solutions, according to a 2022 Splunk survey.

Not to put too fine a point on it, but we think distributed tracing gets a very bad rap for being too complicated and labor-intensive. We’re here to show you three ways you can jumpstart a distributed tracing effort, starting small and expanding as it makes sense. These examples involve only a little code and perhaps a bit of a mindset change. Starting small with distributed tracing can even be fun, because who doesn’t like getting customized results without much work?

A new year has started and I've been pondering my hopes and dreams for the year to come. In the world of SRE, observability is the most prominent pillar of my work. So, I decided to drill into the topic of observability and what I'd like to see happen in the industry in 2023. Rather than focusing on any tool, technology, or methodology, I'lll be exploring concepts that can be broadly applied in any organization.

As a leader in observability and application performance monitoring (APM), New Relic empowers engineers with a data-driven approach to planning, building, deploying, and running great software. Last month, we announced support for New Relic on the CloudZero platform. With this new functionality, customers can gain visibility into their New Relic spend, combine it with any other IT or infrastructure spend, and achieve a complete view of business dimensions — such as products and customers.

Elastic Observability 8.6 introduces a set of capabilities improving production operations through the introduction of host (EC2/GCP compute/Azure compute) observability, application dependency operations views (insights into databases, caches, etc), and a new connector for Opsgenie. These new features allow customers to: Elastic Observability 8.6 is available now on Elastic Cloud — the only hosted Elasticsearch offering to include all of the new features in this latest release.

While IT pros know they need to monitor IT services, they also know it can be the most difficult part of their job. Traditionally, enterprises have cobbled together several disparate monitoring products to address all their monitoring needs – but there are often gaps. Within these gaps, issues are missed, and the possibility of proactive issue resolution becomes nearly impossible.

I recently attended a virtual event and heard the speaker comment, “Relational databases don’t scale.” To my ears, this is about as silly a statement as saying, “No one can eat 26 hot dogs in 12 minutes” right before Kobayashi shows up and eats 50. In my experience, relational databases scale when they’re placed in the hands of someone who knows what they’re doing. Just imagine if Kobayashi was your data architect!

David Calvert is a site reliability engineer working remotely from the south of France. He’s currently focused on observability, reliability, and security aspects of cloud infrastructure. You can find him as dotdc on GitHub and @0xDC_ on Twitter. Over the past three years, I’ve built and operated Kubernetes clusters for two different companies — the first one on-premises, and the second on a public cloud platform for my current job at Powder.

Brick by brick, block by block—if you’ve been with us throughout our Author’s Cut blog series (and if you haven’t, you can go catch up), you’ve seen us build the case for observability from the ground up. We’ve covered structured events, the core analysis loop, and use cases for managing applications in production—and that’s just to start.

Are you just getting started with Cribl Stream? Or maybe you’re well on your way to becoming a certified admin through our Cribl Certified Observability Engineer certification offered by Cribl University. Regardless, using Cribl Stream to send data from one source to many destinations is something you’ll want to try. So if you’re ready, read on!

As we begin the new year, it is customary to reflect and identify areas we can continue to grow in 2023. Whether it’s joining the local gym, starting a new diet, or taking up a new hobby, this time is always full of promise to continually improve. The same can be said for digital businesses of every size and across every vertical. Macroeconomic trends have especially made this time one of reflection for a number of organizations.

For the last few years, the entire networking industry has focused on analytics and mining more and more information out of the network. This makes sense because of all the changes in networking over the last decade. Changes like network overlays, public cloud, applications delivered as a service, and containers mean we need to pay attention to much more diverse information out there.