Operations | Monitoring | ITSM | DevOps | Cloud

February 2024

Decoding .NET8: Unveiling Cloud-Native Observability

The.NET programming language is taking cloud native deployment and observability seriously, and most notably with the recent announcement of.NET Aspire stack unveiled at the recent.NET Conf 2023. In the latest episode of OpenObservability Talks, we reviewed the journey to making.NET a “by default, out of the box observable platform,” as ASP.NET Core creator David Fowler put it.

Is Waiting for the Thaw Unbear-able?

It’s not new news that organizations are producing more data than ever. But, in order to take advantage of this data, it needs to be collected, stored, retained, and then, at some point, analyzed. Most analysis tools also act as the retention point for this data. While this may (at first) appear to be the best option for performance, it quickly creates significant problems. First, those systems were never designed for the scale of today’s growing volume of data, currently at a 28% CAGR.

OpenTelemetry in Production: A Primer

At observIQ, we’re big believers and contributors to the OpenTelemetry project. In 2023, we saw project awareness reach an all-time high as we attended tradeshows like KubeCon and Monitorama. The project’s benefits of flexibility, performance, and vendor agnosticism have been making their rounds; we’ve seen a groundswell of customer interest.

Graylog Parsing Rules and AI Oh My!

In the log aggregation game, the biggest difficulty you face can be setting up parsing rules for your logs. To qualify this statement: simply getting log files into Graylog is easy. Graylog also has out-of-the-box parsing of a wide variety of common log sources, so if your logs fall into one of the many categories of log for which there is either a dedicated Input; a dedicated Illuminate component; or that uses a defined Syslog format; then yes, parsing logs is also easy.

Critical Automation: Anomaly Detection for Application Observability

There’s no debate — in our increasingly AI-driven, lean and data-heavy world, automating key tasks to increase effectiveness and efficiency is the ultimate name of the game. No matter what job you hold today, you’re likely being pushed to not only do more with less, but also perform your work with a tighter focus on specific outcomes and SLOs.

Log Management Made Easy: Top 10 Logs Monitoring Solutions

In contemporary enterprise operations, log management tools have become indispensable for optimizing performance. Among these tools, selecting one with a proficient logs user interface (UI) holds paramount importance. A quality log management tool not only gathers logs but also presents them in a well-organized manner, facilitating easy interpretation for the user.

Beyond Logs: Navigating Entity Behavior in Splunk Platform

Identifying bad actors within your organization often feels like a complicated game of hide and seek. A common comparison is that it's akin to finding a needle in a haystack. So, if the bad actor represents the 'needle' and your organization the 'haystack,' how would you uncover these bad actors? Perhaps the quickest way to find the needle is by burning the haystack. Alternatively, dumping the hay into a pool of water and waiting for the needle to sink to the bottom could also work.

Top 5 Outcomes CIOs Need to Accomplish by 2025: Driving Business Value Through Technology

In January 2024, I published findings from some of my recent research as, “Top 5 Outcomes CIOs Need to Achieve by 2025: Driving Business Value Through Technology.” By focusing on these five key outcomes, CIOs can ensure that their technology investments directly contribute to business growth, resilience, and competitive advantage in the years leading up to 2025.
Sponsored Post

Threat Hunting Frameworks and Methodologies: An Introductory Guide

Establishing an effective cyber threat hunting program is among the top priorities of enterprise security leaders seeking a proactive approach to detecting and counteracting potential threats. Furthermore, implementing a proactive threat hunting program, security teams that leverage formalized frameworks or threat hunting methodologies are far more likely to detect vulnerabilities or in-process malicious activities in their environments than those that do not. However, data from a 2023 threat hunting survey revealed that while 73% of organizations have adopted a defined threat hunting framework, only 38% actually follow it.

The Top 10 Server Monitoring Tools

As organizations and their IT infrastructure become more complex the necessity for effective server monitoring grows. Companies are deciding to operate extensive server networks, utilizing both cloud infrastructure and on-premises data centers due to the ever-increasing demand. Today’s users demand as good as 100% uptime for the services they use, meaning optimal and well–established network connections are vital in order to handle large amounts of users and transactions.

The Next Generation of Papertrail is Here!

We are excited to unveil the next generation of SolarWinds® Papertrail™, SolarWinds Observability® logging. More powerful and faster than ever, the next generation of Papertrail, SolarWinds Observability logging aggregates log data from applications, services, infrastructure, databases, and network devices across both cloud-based and on-premise systems.

Cribl Search and Common Schema: Faster, More Accurate Detections

Are you drowning in data from disparate sources? Are you struggling to analyze it efficiently, sift through different formats, and catch crucial signals? You’re not alone. Cribl Search and Cribl Stream is a powerful combo that lets you unlock insights from vast data volumes – regardless of their source or format. Say goodbye to siloed searches and hello to holistic analysis.

How SOCAR is driving visibility using Sumo Logic

SOCAR needed an observability solution that could parse logs, monitor ephemeral infrastructure in Kubernetes and ensure high visibility into their application, all at a price that fit their budget. Sumo Logic checked all those boxes and has already boosted team collaboration. Learn more about their purchase decision and how they're already making unexpected discoveries.

SigNoz Launch Week - Day 1 - Logs Explorer

Welcome to SigNoz Launch Week 1.0! This is our first launch week, and we’re excited to introduce you to some cool new features in SigNoz. We ship fast but often miss sharing the story behind these features with our community. Launch week for us is an opportunity to share the behind-the-scenes of new features that we have built in the recent past. Our open-source maintainers will share the story on the whats, whys, and hows of new upgrades to SigNoz!

What You Need to Know About ITIL for Service Management

As the person on the front lines, you know that providing the best service possible can be what makes your ITSM organization succeed. Every day, you work to build the relationships that help your organization create value for end-users. However, when you have inefficient processes, you end up having to be the person responding to an upset user.

What is the OpenTelemetry Transform Language (OTTL)?

The OpenTelemetry Transformation Language, or OTTL for short, offers a powerful way to manipulate telemetry data within the OpenTelemetry Collector. It can be leveraged in conjunction with OpenTelemetry processors (such as filter, routing, and transform), core components of the OpenTelemetry Collector. It caters to a range of tasks from simple alterations to complex changes.

Build better Service Level Objectives (SLOs) from logs and metrics

In today's digital landscape, applications are at the heart of both our personal and professional lives. We've grown accustomed to these applications being perpetually available and responsive. This expectation places a significant burden on the shoulders of developers and operations teams.

Data Here, Data There, Data Everywhere: the Benefits of Routing Data With Cribl

As an organization, you likely have many choices on where to store, analyze, and correlate your data. Those choices may change or iterate over time, so having an easy way to route data is needed. Enter Cribl Stream, which can route your data where it needs to go and save some effort, time, and money. It can help with organizational-wide initiatives like migrations and consolidations but can also help with smaller-scale initiatives and your day-to-day tasks of simply getting data in.

The Importance of DevOps Analytics

Traditional software development and infrastructure management module for production and service has been overtaken by the quicker-paced delivery of services and applications, DevOps. This outperformance by DevOps in response to the traditional approach has led to numerous organizations making DevOps a fundamental part of the company.

How Cribl Stream Can Enhance Digital Operational Resilience Under DORA within Financial Services

In the swiftly changing digital realm of the finance and insurance sectors, sustaining operational resilience while complying with rigorous regulatory mandates is paramount. The Digital Operational Resilience Act (DORA) marks a significant regulatory milestone designed to ensure entities within the financial services sector are equipped to withstand, respond to, and recover from all types of ICT (Information and Communication Technology) related disruptions and threats.

The Leading Reporting Dashboard Examples

Dashboards provide an enhanced view of your most critical business metrics. With the majority utilizing both real-time and historical data, they enable you to promptly respond to current trends as well as accurately forecast for the future. Also, reporting dashboards excel when compared to static reports, in regards to presenting data and objectives to stakeholders.

Capturing Security and Observability Data From Oracle Cloud

A couple of years ago, I wrote another blog on how Oracle Cloud Infrastructure (OCI) Object Storage can be used as a data lake since it has an Amazon S3-compliant API. Since then, I’ve also fielded several requests to capture logs from OCI Services and send them through Cribl Stream for optimization and routing to multiple destinations. There are two primary methods to achieve this.

Why is Log Monitoring Considered to be Important?

Log monitoring has become crucial nowadays as more than 90% of organizations use cloud services, containers, and other technologies to stay ahead of their competitors. This excessive adaption of the latest technologies and services is great for businesses but it also makes everything a bit more complex. Consequently, the volume, velocity, and diversity of logs rise exponentially as a result of this complexity.

Understanding AI search algorithms

Artificial intelligence tools are everywhere, and it’s no mystery why. They can carry out a huge variety of tasks and find the solutions to many everyday problems. But these apps are only as good as their AI search algorithm. In simple terms, an AI search algorithm is the decision-making formula an AI tool uses to find the optimal solution to your specific problem. Search algorithms may make trade-offs between speed, relevance, or another weighted factor.

Fluentd vs. Fluent Bit: A Comparison

Recently, I came across the quote, "The goal is to transform data into information and information into insights." This statement emphasizes the significance of data (logs) and the responsibilities associated with software applications for robust data management, particularly log management. Effective log management is essential for performance optimization, troubleshooting, ensuring strong security, and maintaining compliance.

5 AI search trends impacting developers in 2024

After an incredibly fast-moving 2023, what does the future hold for AI and search? Conversational generative AI leapt into the public consciousness over the past year, and organizations scrambled to define their strategy for capitalizing on the trend. AI-boosted relevance is reshaping the way users experience search — and elevating their expectations for the quality of the interaction.

Evolving Your Career Path in Tech: Insights and Strategies for Success

Discover the unique career paths of our panelists in the tech industry. Overcoming imposter syndrome, setting and achieving goals, and navigating the nuances of company culture can pose challenges in one's professional journey. However, these women will share their stories of resilience, overcoming obstacles, and self-advocacy that propelled them to where they are today.

Enhancing Log Analytics in Loki with Cribl Stream

First, when I mention Loki, I’m not talking about one of my favorite TV shows to binge-watch or the lead character played by Tom Hiddleston, who has arguably become one of my favorite characters in the Marvel universe. I’m talking about the Loki, which is a highly available, cost-effective log aggregation system that was inspired by Prometheus. While Prometheus is focused on metrics, Loki is focused on collection of logs.

Building Your Own Observability Solution vs Implementing a SaaS Solution

Observability is a key component of modern applications, especially highly complex ones with multiple containers, cloud infrastructure, and numerous data sources. You can implement observability in two ways: build your own observability solution or use a homegrown alternative like Coralogix.

Track events in real time: Enhance monitoring with proactive log analysis

Preventing issues through proactive log analysis is more advantageous than reacting to problems with troubleshooting when they occur. Logs can act as a powerful source for proactive monitoring, and configuring the right alerts can ensure that you are notified about critical events in advance. In this blog post, we'll unveil a few suggestions for optimizing log-based alerting to enhance incident management and achieve operational excellence.

Thou Shall Pass! Troubleshooting Common Amazon S3 Errors in Cribl Stream

Data lakes are everywhere! With data volumes increasing, cost-effective storage is becoming a greater need. With Cribl Stream, you can route data to an Amazon S3 data lake and replay or search that data at rest. But nothing is more frustrating than something not working and those blasted error logs that pop up. In this blog, some common errors for your S3 sources or destinations are highlighted, and some potential root causes and solutions are highlighted.

Best Log Monitoring Tools

Log monitoring is a fundamental practice in the system administration and cybersecurity, playing a pivotal role in maintaining the health and security of computer systems. At its core, log monitoring revolves around the scrutiny of log files generated by diverse software applications, operating systems, and servers. These log files serve as detailed records, containing crucial information about system events, errors, and user activities.

Are You Forensic Ready?

In the landscape of everyday operations, the concept of forensic readiness may often linger unnoticed in the background. When a crisis strikes, be it a major system outage or a security breach. The importance of being forensic ready as part of your overall digital resiliency strategy suddenly becomes evident. That’s the moment you realize it’s necessary for a thorough investigation. The findings enable you to have an effective response and proportionate mitigative actions.

Greater Control Over Windows Events for Qradar: Why Windows Events Matter

Windows events provide a wealth of security-relevant information, especially when they are correlated and analyzed within a SIEM like IBM Qradar. Whether you rely on MITRE ATT&CK, NIST, or another security framework, Windows Events are likely one of your higher volumes (EPS – Events Per Second) and represent your largest-sized events (Gigs per day – Storage and Archive).

The Role of Observability in Telecoms

The rapid growth of 5G technology and expanse of the Telecoms industry has created the need for these organizations to implement effective data-driven decisions, to enable the future profitability of their companies. This raises the challenge of analyzing data from various sources across complex networks to derive insights and ultimately decision making.

Get Swept Off Your Feet by Cribl Stream 4.5: Converting Dimensional Metrics to the OpenTelemetry Protocol Format with the OTLP Metrics Function

In the dynamic world of observability and analytics, everyone’s looking for smarter, more efficient, and interoperable ways to handle their data. That’s where Cribl steps in, bringing you an exciting update to our product lineup. We’re thrilled to introduce the OTLP Metrics Function to Cribl Stream 4.5! This Function converts metrics into the OpenTelemetry Protocol (OTLP) format with ease!

Advanced Log File Monitoring Strategies on Microsoft SCOM and Azure Monitor

This technical whitepaper delves into the intricacies and benefits of advanced log file monitoring, showcasing its pivotal role in modern IT infrastructure management. We explore the fundamental principles of log file monitoring, discuss the challenges associated with traditional approaches, and highlight the advantages of adopting advanced techniques.

Don't Slow Your Roll: Controlling Your Qradar Data Flow

IBM Qradar is a Security Incident and Event Manager (SIEM) trusted by many organizations to provide threat detection, threat hunting, and alerting capabilities. Qradar SIEM is often integrated with complementary IBM tools or enhanced with extensions to meet the needs of organizations that wish to mitigate their risks.

Testing logging code with Microsoft.Extensions.Logging and FakeLogger

Unit testing is most often used for testing business logic. But what if you want to ensure that your code logs important messages to your log store? In this post, I'll introduce you to FakeLogger and how it can be used to test logging code when using Microsoft.Extensions.Logging and the ILogger interface. So, let's start by discussing why to even unit-test logging code. Adding good logging to your code is an often forgotten or down-prioritized practice.

Home Shopping Europe (HSE) increases customer satisfaction using Elasticsearch on AWS

Home Shopping Europe (HSE), a prominent player in the European live commerce sector, has revolutionized its customer experience by leveraging Elastic on AWS. Elastic's AI and ML features in Elasticsearch deliver accurate and relevant search results. This enhancement has not only elevated click-through rates by 4% but has also significantly reduced maintenance time by 42%, marking a pivotal shift for HSE's e-commerce business.

Aggregate Data in Cribl Stream to Optimize Your SIEM Data and Its Performance

Cribl Stream offers different ways to optimize data, such as: In this blog, I will focus on the Aggregation use case using the Aggregations function and how you can practically use the Aggregations function to format the output in different ways.

How to Build Dashboards

Reporting and analytics dashboards provide enhanced visibility into your data and the ability to view your most critical metrics via a single source of truth. By using dashboards, your team can easily highlight issues or areas of concern and promptly begin addressing them utilizing the real-time data that a dashboard provides. As well as this they can be utilized to drive data-driven decisions for your organization, enabling greater accuracy for decision-making to drive growth.

Building Large-Scale User Behavior Analytics: Data Validation and Model Monitoring

As the demands of our customers continue to rise, Splunk User Behavior Analytics (UBA) V5.3 now boasts an increased ingesting rate up to 160K EPS from Splunk Enterprise to a 20-node large deployment. This scalability improvement facilitates support for 750K user accounts, 1 million devices, and 64 data sources.

Latest Top 11 Log Monitoring Tools [Includes Open-Source]

For any software company, a log monitoring tool is a must for collecting, storing, and providing a centralized view of all logs from different applications and hosts for faster anomaly detection, incident resolution, and troubleshooting. They can also help detect security threats and provide audit trails. They are effective in capacity planning, decision-making, and ensuring optimized performance.

Elastic APM for iOS and Android Native apps

Elastic APM for native apps provides auto-instrumentation of outgoing HTTP requests and view-loads, captures custom events, errors, and crashes, and includes pre-built dashboards for data analysis and troubleshooting purposes Elastic® APM for iOS and Android native apps is generally available in the stack release v8.12. The Elastic iOS and Android APM agents are open-source and have been developed on-top, i.e., as a distribution of the OpenTelemetry Swift and Android SDK/API, respectively.

The Top 8 Network Monitoring Tools

Network Monitoring is a process that supplies the information and data that network administrators need to determine, in real-time, the status of their network and if it's running optimally. This enables these administrators to work proactively to highlight deficiencies, enhance efficiency, and more. By utilizing network monitoring you can attain complete visibility into their network.

This Month in Datadog: Dynamic Instrumentation, Log Pipeline Scanner, Network Device map, and more

Datadog is constantly elevating the approach to cloud monitoring and security. This Month in Datadog updates you on our newest product features, announcements, resources, and events. This month, we put the Spotlight on Dynamic Instrumentation..

Better Practices for Connecting Cribl Stream to Many Splunk Indexers

Cribl Stream and Cribl Edge can send data to Splunk in several different ways. In this blog post, we’ll focus on the common scenario where you want to connect Cribl Stream’s Splunk Load Balanced Destination to many Splunk Indexers at once. (We’ll talk about Cribl Stream, but what we say applies to Cribl Edge, too.) Cribl Destinations settings default to reasonable values. Sometimes Cribl Support recommends changing those values for better results in a given situation.

Behind the Scenes with the Splunk Brand Refresh

Splunk had just celebrated its 20th anniversary. The business was growing. Customers were loyal. So why would we consider refreshing our brand? The answer is simple, if you aren’t growing, you’re declining. Just like people, brands need to adapt and grow so they stay relevant. For us, part of our growth was reaching new audiences and launching new products. Which meant as brand stewards, we needed to update our brand to better connect with these new opportunities.

Taming Tetragon With Cribl.Cloud

Did you know you can deploy Tetragon and parse high-volume logs with Cribl Edge? It’s true! Tetragon integrates seamlessly with Cribl Edge. This combination enhances monitoring capabilities in Linux environments. Have your cake and eat it, too. With a combined Cribl and Isovalent solution, you can deliver deep insights into your workloads, optimizing for your specific operational requirements with zero loss of data fidelity.

Universal Profiling: Detecting CO2 and energy efficiency

A while ago, we posted a blog that detailed how we imported over 4 billion chess games with speed using Python and optimized the code leveraging our Universal ProfilingTM. This was based on Elastic Stack running on version 8.9. We are now on 8.12, and it is time to do a second part that shows how easy it is to observe compiled languages and how Elastic®’s Universal Profiling can help you determine the benefit of a rewrite, both from a cost and environmental friendliness angle.

What is the Benefit of Including Security with Your Observability Strategy?

Observability strategies are needed to ensure stable and performant applications, especially when complex distributed environments back them. Large volumes of observability data are collected to support automatic insights into these areas of applications. Logs, metrics, and traces are the three pillars of observability that feed these insights. Security data is often isolated instead of combined with data collected by existing observability tools.

6 Benefits of an AI-Powered Observability Pipeline

Observability Pipelines have become vital tools for DevOps and Security teams to manage, control, store, route, and optimize telemetry data analyzed by Security Information and Event Management (SIEM), Application Performance Monitoring (APM), and Log management platforms. These teams spend hours every week trying to fit an increasingly large volume of data into the same size box.

Client-side Logging: Optimize Performance and Enhance the User Experience

Performance optimization is crucial when developing user-centric applications. To achieve better performance, it is essential to maintain effective log management. Client-side (user) logging is vital in driving website traffic or increasing user engagement with your applications. After deploying an application or a web browser, client-side information, such as user behavior, events, and errors, is not stored by default.

Improve Cloud Visibility with JFrog's SaaS Log Streamer

The beauty of deploying SaaS-based applications is that you don’t have to worry about building the infrastructure, hiring engineers to maintain it, staying on top of upgrades or worry about application security. Indeed, these are some of the main benefits you get by using a SaaS offering. However, the world of software is full of trade-offs, so, what do you lose out on?

Monitoring Cribl Stream with Elasticsearch

Are you managing a Cribl environment? We love that for you; you’re at the forefront of complex data orchestration. As the steward of this dynamic data ecosystem, you have to manage and optimize the flow of information from diverse sources. As data volumes grow, the struggle gets even more real. No worries, though. You’ve got Cribl Stream. Monitoring Stream is critical.

A Beginner's Guide to Structured Logging

Structured logging is a methodical approach to log management in software development, often utilizing JSON or key-value pairs. This method enhances the comprehensibility and analytical efficiency of log data, particularly in complex and distributed system environments. Unlike unstructured logs, which lack a defined format, structured logs adhere to a standardized layout, facilitating streamlined analysis and troubleshooting.

A Practical Guide to Logging in Microservices [Includes Best Practices]

Microservices logging is the practice of tracking and recording the activities of specific services in a distributed microservices architecture. Logging is an important aspect of any software system, and it is more critical for a microservices architecture as there are many small, independent services interacting with each other.

Let's Put on a Show With Cribl's Search Sandbox!

Remember when you were a kid and your school put on a production of the latest grade school drama? Maybe you didn’t get the lead role, but it was fun to put on (or watch) the show. Search Sandboxes are just like that! Except you get to be the stage manager when searching data. And Search Sandboxes offer you everything you need to make it an all-star performance.

Micro Lesson: A Log's Journey

Meet Rick Jury, Senior Technical Account Manager at Sumo Logic. In this video, Rick talks about the ingestion pipeline and the journey that a log message takes from collection into the Sumo platform, and considerations for administrators around the ingestion pipeline. You will be excited to see how this translates into a search, turning a raw event into a schema and then into actual insights.

Visualize Sumo Logic metrics and logs with Grafana: Introducing the Sumo Logic Enterprise plugin

We are thrilled to announce the addition of a powerful new Enterprise plugin in the Grafana ecosystem: the Sumo Logic Enterprise data source plugin for Grafana. You can now easily connect Sumo Logic to your Grafana instance and correlate your log data with telemetry from all your data sources in one unified Grafana dashboard.

Delivering Value with a Flat Budget

Join us for an important conversation with Cribl's Ed Bailey and Jackie McGuire, as we navigate the intricate balance of maximizing organizational value with a constrained budget. In today's challenging economic climate, where maintaining operations often means minimal to no additional spending, adaptive strategies become crucial. This is more than just a best-case scenario; it's a necessary approach for business resilience. Ed and Jackie will share innovative ideas and strategies to help leaders skillfully manage tight budgets while delivering significant value to their organizations.

Dashboard Studio Feature Highlights in Splunk Enterprise 9.2

With every major Splunk Enterprise release, we level up your dashboarding experience so that you can visualize and take action on your data fast. In Splunk Enterprise 9.2, we are bringing the experience across Classic (SimpleXML) dashboards and Dashboard Studio closer together and weaving in Dashboard Studio features from the two most recent Splunk Cloud Platform releases. This blog post covers the major dashboarding features included in Splunk Enterprise 9.2.

Mastering Firewall Logs - Part 2

As a pivotal element within your networking configuration, logs generated by Network Firewalls hold immense importance from both security and compliance standpoints. These logs serve as a source of valuable information, encompassing records of network traffic details like source and destination IP addresses, ports, protocols, timestamps, and the actions (e.g., allowed or denied) taken by the firewall for each connection or packet.