The heart of problem-solving is to outline and define the problem effectively, therefore, if you aren’t aware of what the problem is then it’s particularly challenging to accurately problem solve. This is where root cause analysis tools come in, root cause analysis software and tools are designed to identify the the cause of the issue to aid your ability to rectify the problem effectively.

Thanks for joining me for Part 3 of “The concise guide to Grafana Loki,” a series of blog posts that takes a closer look at best practices for various aspects of using the log aggregation system. Today’s post is my holiday present for all the folks out there running Loki who would like to get the most query performance they can out of their cluster.

As an engineer, you know your company’s problems, and you know what to do about them. However, being heard within your organization and funding a project can be challenging. Top executives might not understand your job’s ins and outs of the tools you need to do it well. Still, you need people holding the purse strings to understand why investing in your idea is brilliant.

With constantly decreasing user attention spans, ensuring a seamless user experience has become a priority for all digital businesses. Users who encounter minimal application disruptions and responsive interactions will likely stay engaged and loyal to your product. And that’s exactly what RUM or Real User Monitoring tools such as Coralogix’s RUM solution offer.

“Hasn’t everyone already migrated to the cloud?” is a question you might be considering now. For many businesses – sure, they’ve migrated workloads and operations to the major cloud providers like Amazon Web Service, Google Cloud Platform, and Microsoft Azure. Still, many businesses have just now worked through their due diligence and scalability concerns. While many businesses are “fully cloud,” there are just as many yet to migrate.

Understanding the expected behavior of the Splunk Load Balanced (Splunk LB) Destination when Splunk indexers are blocking involves complex logic. While existing documentation provides details into how the load-balancing algorithm works, this blog post dives into how a Splunk LB Destination sends events downstream and explains the intricacies of blocking vs. queuing when multiple targets (i.e., indexers) are involved.

There are several reasons for creating a highly efficient and performant database in the current web era. RocksDB is an embedded key-value store designed for efficient data storage and retrieval. It is an open-source database engine developed by Facebook, which builds upon the strengths of LevelDB while incorporating several enhancements for durability, scalability, and performance.

With 2023 drawing to a close, the final OpenObservability Talks of the year focused on what happened this year in open source, DevOps, observability and more, with an eye towards the future. I was delighted to be joined by a special guest, Kelsey Hightower, a renowned figure in the tech community, especially known for his contributions to the Kubernetes ecosystem.

Welcome to Part 2 of the “Concise guide to Loki,” a multi-part series where I cover some of the most important topics around our favorite logging database: Grafana Loki. As I reflect on the fifth anniversary of Loki, it felt like a good opportunity to summarize some of the important parts of how it works, how it’s built, how to run it, etc. And as the name of the series suggests, I’m doing it as concisely as I can.

Large-scale organizations typically collect and manage millions of logs a day from various services. Within these orgs, many different teams may set up processing pipelines to modify and enrich logs for security monitoring, compliance audits, and DevOps. Datadog Log Pipeline let you ingest logs from your entire stack, parse and enrich them with contextual information, add tags for usage attribution, generate metrics, and quickly identify log anomalies.

Audit finds that there are 1,200+ government AI use cases in development and in use today.

For businesses reliant on customers’ positive digital experiences to achieve their goals, the seamless operation of cloud applications and infrastructure is paramount for financial success. Observability holds a pivotal role in modern enterprises, offering critical insights into your IT system’s health and performance. However, persistent issues of complexity and high costs have plagued the observability landscape.

Elastic is transforming the log experience to meet the needs of modern workflows In the absence of other observability signals, generally everything in your infrastructure (hardware, software, and services) emits log lines. Logs, however, are often structured at a developer’s whim and, first and foremost, serve the developer’s needs (e.g., debugging).

The European Union’s new legislation is the first of its kind — and has global reach On December 8, 2023, the European Union made a significant step in digital governance by introducing the first set of comprehensive artificial intelligence (AI) regulations. This legislation, poised for a European Parliament vote by early 2024, is first out of the gate in regulating AI.

While tuning isn’t strictly required, Cribl Support frequently encounters users who are having trouble getting data into Stream from Splunk forwarders. More often than not, this is a performance issue that results in the forwarders getting blocked by Stream. When they encounter this situation, customers often ask: How do I get data into Stream from my Splunk forwarders as efficiently as possible? The answer is proper tuning!

Cribl Stream is awesome at routing your server logs and making your job easier, but could it help you outside of work and potentially make your personal life easier? The short answer is: Yes. I’ve personally used Stream to build a notification system to inform me when certain products go on sale or when fully booked appointments become available. In this blog, I’m going to take this a step further and show you how to.

Observe is a SaaS based observability tool built on Snowflake. It offers a graph-style approach to observability data, claiming that this makes it easier to correlate data in a seamless fashion. Let’s see how Observe compares to Coralogix.

Last week, I attended the Gartner IT Infrastructure, Operations & Cloud Strategies Conference (IOCS). Gartner IOCS is my favorite conference every year because of the quality and level of the presentations. Gartner analysts deliver most sessions and put a lot of effort into the presentations and supporting research. ‍ I’d like to highlight two sessions that I found to be very informative.

Monitoring distributed systems means collecting data from various sources, including servers, containers, and applications. In large organizations, this data distribution makes it harder to get a single view of the performance of their entire system. OpenTelemetry helps you streamline your full-stack observability efforts by giving you a single, universal format for collecting and sending telemetry data. Thus, OpenTelemetry makes improving performance and troubleshooting issues easier for teams.

In this conversation, Cribl’s Carley Rosato talks to Aflac’s Shawn Cannon about his role as a Threat Management Consultant, and how he manages their SIEM environment, brings in new data as needed, and works to improve the ingestion process. Our customers are always coming up with new and exciting ways to implement Cribl tools — importing a 34 million-row CSV file into Redis and enriching events in Splunk might be one of the most impressive we’ve seen so far.

As we continue to navigate the ongoing evolution of the observability landscape, Logz.io is constantly striving to provide our customers with the advanced platform capabilities needed to make sense of their increasingly complex environments. Sometimes that means taking a new approach to long-standing practices.

We're excited to announce AppSignal support for Vector logs and metrics! AppSignal's Vector support allows you to expand your monitoring horizons beyond our standard language integrations, making it possible to leverage AppSignal to both monitor the performance and manage the logs of components of your stack that fall outside a standard application. With Vector, you can use AppSignal to monitor how your databases and Kubernetes clusters perform and metrics from many other sources.

Google Workspace is a robust set of productivity applications with billions of users and millions of paying organizations. These include small mom-and-pop shops and the largest enterprises. Google provides the Google Reports API, “a RESTful API you can use to access information about the Google Workspace activities of your users.” This data is critical for establishing a solid security posture.

Five years ago today, Grafana Loki was introduced to the world on the KubeconNA 2018 stage when David Kaltschmidt, now a Senior Director of Engineering at Grafana Labs, clicked the button to make the Loki repo public live in front of the sold-out crowd. At the time, Loki was a prototype: We bolted together Grafana as a UI, Cortex internals, and Prometheus labels to find out if there was a need for a new open source tool to manage logs.

In the US, a recurring news topic is the state of the federal budget – and if we’ll get one signed. Government budgets have hundreds of thousands of line items; each bickered over to gain or lose political capital with one group or another. However, most government budgets aren’t up for debate. Only about 30% of the US federal budget is discretionary or flexible. Nearly two-thirds, or 63%, is mandatory spending required due to prior commitments.

Data is growing, and we are being asked to search larger and larger amounts of data. This puts larger and larger demands on Search resources. Reading all the data to find matching events is muscling through the data. Wouldn’t it be more efficient to be able to do filtering before reading the data? Cribl Search does precisely that by leveraging Parquet Pushdowns.

As we reflect on AWS re:Invent 2023, the Coralogix team is invigorated by the incredible response and feedback we received from the thousands of participants who visited our booth. It was clear that a recurring theme among companies is the need for an observability solution that not only scales affordably with increasing data volumes but is also at the forefront of innovation. Coralogix stands out as the ideal match for these requirements.

In the ever-evolving world of data analysis, the ability to interact directly with live API endpoints is a significant advancement for practitioners. Cribl Search now offers this capability, enhancing your data analysis toolkit. This new feature allows you to gain broader visibility into the periphery of your infrastructure, enabling a more comprehensive analysis of user journeys and operational trends.

Around 70% of companies experienced cyberattacks in the past year. With this increase in cyberattacks, the importance of log management in IT security has also increased over the years. That’s the reason why small and enterprise businesses have started to invest in log management tools to protect their businesses from cybersecurity breaches.

As applications in the cloud become more distributed and complex, the Mean Time To Resolution (MTTR) for production issues is getting longer. Modern systems are built with hundreds of distinct, ephemeral, and interconnected cloud components, which can make it exceptionally hard for engineers to understand the current state of their applications, what problems are impacting customers, and why those problems are occurring.

Years before founding Logz.io, I was a software engineer, working with various tools to ensure my products and services performed correctly. There were few tools I dreaded using more than application performance management (APM), and I know that I’m not alone. I hated traditional APM. It’s heavy. It’s hard to implement. It’s expensive. It takes a very long time to derive business value.

Last week, I attended the Amazon Web Services (AWS) re:Invent conference in Las Vegas, NV, with 50,000+ others. It was quite a busy week with several keynotes, announcements, and many sessions. While the hot topic at re:Invent was generative AI, I’ll focus my blog post on a few customer sessions I attended around observability: Stripe, Capital One, and McDonald’s. ‍

In the world of data management, Cribl offers various methods to enhance data using the Lookup Function and many C.Lookup Expressions. While Cribl’s documentation is comprehensive, practical examples are often the most effective learning tools. That’s why we’ve introduced the new Lookup Examples Pack.

In the ever-evolving landscape of data integration and architecture, organizations grapple with many challenges, from controlling exponentially growing observability data to the complexities driven by hybrid clouds, data migrations, integration of new AI/ML services, and the need for swift time-to-market strategies.

Cribl Stream is a real-time security and observability data processing pipeline that can be used to collect, transform, enrich, reduce, redact, and route data from a variety of sources to a variety of destinations. One of the popular destinations for Cribl users is Elastic SIEM. This blog post will walk you through the steps on how to set up Cribl Stream to normalize and forward data to use with Elastic Security for SIEM.

There is more data available to us than ever. Storing this data is important — but deciding on the right type of data storage solution is not so clear. This article explores two primary types of big data storage: data lakes and data warehouses. We’ll examine the benefits of each, then discuss the key differences between a data lake and a data warehouse, so you can decide on the best approach for your business.

In this blog series, we’ll explore how Cribl Stream can leverage your existing cross-domain solution (CDS) to easily collect and send your log and metric data between disparate security domains or across air-gapped networks. The goal is to retain as much fidelity of the data as possible, deduplicating processes and simplifying management efforts.

While we built Grafana Loki as an open source log aggregation system that is cost effective and easy to operate, let’s face it: sometimes there is no time or bandwidth to mess around with self-managing and self-hosting. Luckily there’s the fully managed Grafana Cloud observability stack for log management. “Grafana Cloud is a no-BS platform. The engineering costs of hosting it ourselves would be much higher," says Jameel Al-Aziz, a software architect at Paradigm.

Circles X uses DORA DevOps best practices to build the first telco-as-a-service in Indonesia, helping partners to launch a digital telco.

In today’s hybrid cloud era, the volume and diversity of log data have exploded, which makes managing them ever so challenging. IT teams need to conquer the gush of logs by providing context whilst having an effective log management strategy. Without a powerful log management solution, it all becomes too cumbersome. And even if you do get your hand around a good log management platform, you’ll find yourself stuck with hefty licensing costs and impractical compliance issues.

Recently, I sat down with Adelaide O’Brien, research vice president at IDC Government Insights, to discuss the current and future state of generative AI in the public sector worldwide. The full conversation is available to view on demand, but I also wanted to highlight some of the takeaways from the discussion.

In the following comparison table, we will provide you with an extensive guide designed to enable a detailed assessment of Cassandra and OpenSearch. This comparison aims to supply an in-depth exploration of multiple aspects of these two database systems, providing you with the insights required to make informed decisions tailored to your specific use case.

Transunion is an American consumer credit reporting agency that operates in over 30 countries. They use Cribl Stream to aggregate and route regional data into a centralized hub, presenting it in a single dashboard that admins can use to interpret the overall health of their system. Watch the full video on YouTube or below to see Transunion’s Steve Koelpin and Don Reilly walk through this use case.