If you’re a software developer, then you understand how vital application logging is in software development and a critical part of logging is something called logging levels. Log entries generally contain essential information—such as a timestamp, a message, and sometimes additional stuff like an exception’s stack trace. Those pieces of information are useful because they allow someone reading the log entry to understand how the application behaved in production.

Kafka and the ELK Stack — usually these two are part of the same architectural solution, Kafka acting as a buffer in front of Logstash to ensure resiliency. This article explores a different combination — using the ELK Stack to collect and analyze Kafka logs. As explained in a previous post, Kafka plays a key role in our architecture. As such, we’ve constructed a monitoring system to ensure data is flowing through the pipelines as expected.

At Honeycomb, we’re pretty engineering-focused. We think our product is something every developer can benefit from, and we’re working to build something we would want to (and do!) use every day to make our own jobs easier.

Let’s start with the happy ending — after a long search, we managed to identify a Netty memory leak in one of our log listeners and were able to troubleshoot and fix the issue on time before the service crashed.