Operations | Monitoring | ITSM | DevOps | Cloud

Kafka Performance Crisis: How We Scaled OpenTelemetry Log Ingestion by 150%

When your telemetry pipeline starts falling behind, the countdown to production impact has already begun. One Bindplane customer operating a large-scale log ingestion pipeline built on the OpenTelemetry Collector and Kafka hit that breaking point. Instead of keeping pace with incoming data, their pipeline was ingesting just 12,000 events per second (EPS) per partition/collector—and this Kafka topic had 16 partitions. In aggregate, that was roughly 192K EPS.

Resilience with Zero Data Loss in High-Volume Telemetry Pipelines with OpenTelemetry and Bindplane

This was the problem one Bindplane customer had with processing enormous S3-stored log files. Our engineering team tackled the problem head-on, enhancing the S3 event receiver with offset tracking and chaos testing methodologies.

How to Build Resilient Telemetry Pipelines with the OpenTelemetry Collector: High Availability and Gateway Architecture

Let’s bring that back. Today you’ll learn how to configure high availability for the OpenTelemetry Collector so you don’t lose telemetry during node failures, rolling upgrades, or traffic spikes. The guide covers both Docker and Kubernetes samples with hands-on demos of configs. But first, let’s lay some groundwork.

AI + Dark Mode: Introducing AI-Powered Insights and The Long Awaited Dark Mode

Join the live stream at 11 am ET, here. Launch Week’s Friday drop delivers two of the most-requested upgrades we’ve ever shipped: Together, they turn Bindplane into a cooler , and smarter , place to manage observability and SecOps telemetry. A full suite of extensive AI features will be rolling out over the coming weeks. This is just the beginning!

Blueprints: Ready-Made Processor Bundles For Your Telemetry Pipelines

We’ve noticed a lot of our customers spend countless hours building and configuring processors. Either parsing JSON, standardizing log formats, normalizing timestamps, masking PII, de-duplicating logs, the list never ends. Most work revolves around recreating the same processor bundles in multiple processor nodes. Bindplane’s new Blueprints solves that boring, repetitive work by providing pre-built processor bundles you can drop into any pipeline with a single click.

Scaling Observability: How We Designed Bindplane to Manage 1,000,000 OpenTelemetry Collectors

Join the live stream at 11 am ET, here. Platform teams tend to start with just one, or in some cases a handful of OpenTelemetry (OTel) Collectors usually running in gateway mode. They then embrace the benefit of a vendor-neutral, standardized, telemetry collector for unified logs, metrics, and traces.

Your Collector, Your Rules: Introducing BYOC and the OpenTelemetry Distribution Builder

Join the live stream at 11 am ET, here. OpenTelemetry’s super-power has always been: Choice. Yet, most observability vendors still insist you run their collector. Today we’re removing that last point of friction. With Bring Your Own Collector (BYOC), Bindplane now accepts any upstream-compatible build, recognizes exactly which receivers, processors, and exporters it contains, and adapts the UI and configuration workflow on the fly.

Unify telemetry, own your pipeline: New integrations for Windows, Network Telemetry, and Cloud Storage

Today, we're expanding on the integrations front, and launching new integrations for Windows events, network telemetry, and cloud storage. Here's a quick tour of what's new and why it matters.

Strategic Windows Event Routing with Bindplane

Windows event logs can provide valuable insight into day-to-day operations and potential security issues. But making sense of that data—and getting it to the right place without overloading your systems or driving up costs—takes some planning. Bindplane helps with this by providing a flexible way to collect, process, and route Windows events. It’s designed to support security and compliance needs without adding unnecessary complexity.

Serverless Monitoring In The Cloud With Bindplane and OpenTelemetry

Almost two years ago I wrote the first installment of what was supposed to be a 3 part series on Serverless Monitoring. Parts two and three never materialized. Today, however, I am revisiting that original idea and expanding upon it. I hope to succeed this time in making it a full three-part series. For this first installment (Revisited), I will again work with Google Cloud Run to monitor MongoDB Atlas.