Hello, Mattermost community! I’m Andrew Zigler, a developer advocate here at Mattermost, and I’ve recently had the privilege of representing our open source research at two great cybersecurity community conferences: BSides Barcelona 2024 and AppSec SoCal 2024.

Efficient communication between applications is essential for seamless operations in modern IT environments. IBM MQ, robust messaging middleware, plays a pivotal role in facilitating this. Monitoring IBM MQ is crucial for enhancing communication efficiency and enables organizations to optimize performance, identify potential issues before they escalate, and ensure reliable message delivery.

Apache Kafka has become an essential component in data streaming and processing architectures due to its high throughput and scalability. However, as organizations scale up their Kafka usage, they often encounter challenges such as partition rebalancing across different brokers. This imbalance can lead to significant issues, including overloaded partitions that jam traffic, affecting performance and reliability.

Working in the observability and monitoring space for the last few years, we have had multiple users complain about the lack of detailed monitoring for messaging queues and Kafka in particular. Especially with the coming of instrumentation standards like OpenTelemetry, we thought there must a better way to solve this. We dived deeper into the problem and were trying to understand what better can be done here to make understanding and remediating issues in messaging systems much easier.