The days when you could simply SSH into a server and perform a fancy grep are long gone. If you’re reading this article, chances are either you are looking to move from that obsolete approach to a centralized logging approach with a log management tool, or you are looking for an alternative log management tool to replace your existing solution. Problem is, there are so many different tools out there, making a choice can be overwhelming. So how do you pick the right solution?

In a previous post, we introduced a new integration with Microsoft Azure that makes it easy to ship Azure logs and metrics into Logz.io using a ready-made deployment template. Once in Logz.io, this data can be analyzed using the advanced analytics tools Logz.io has to offer — you can query the data, create visualizations and dashboards, and create alerts to get notified when something out of the ordinary occurs.

There’s no denying it — Kubernetes has become the de-facto industry standard for container orchestration. In 2018, AWS, Oracle, Microsoft, VMware and Pivotal all joined the CNCF as part of jumping on the Kubernetes bandwagon. This adoption by enterprise giants is coupled by a meteoric rise in usage and popularity. Yet despite all of this, the simple truth is that Kubernetes is hard.

If you’ve been following Elasticsearch-related news over the past few months, you’ve most likely heard about a series of cases in which sensitive data stored in Elasticsearch clusters was exposed. Here’s a recap just in case — Equifax, CITI, AIESEC to name just a few.

Suricata is an open source threat detection system. Initially released by the Open Information Security Foundation (OISF) in 2010, Suricata can act both as an intrusion detection system (IDS), and intrusion prevention system (IPS), or be used for network security monitoring.

Many of my fellow engineers ask me what it means to be an SRE (Site Reliability Engineer). When I tell them it’s a type of DevOps engineer, they get a glazed look in their eyes and then ask what a DevOps engineer is. I then find myself googling both job titles and reading twelve very different definitions until I reach the conclusion that these definitions vary wildly from company to company and from team to team.

In a previous article, we explained the importance of monitoring the performance of your servers. Keeping tabs on metrics such as CPU, memory, disk usage, uptime, network traffic and swap usage will help you gauge the general health of your environment as well as provide the context you need to troubleshoot and solve production issues.

Kafka and the ELK Stack — usually these two are part of the same architectural solution, Kafka acting as a buffer in front of Logstash to ensure resiliency. This article explores a different combination — using the ELK Stack to collect and analyze Kafka logs. As explained in a previous post, Kafka plays a key role in our architecture. As such, we’ve constructed a monitoring system to ensure data is flowing through the pipelines as expected.

Data breaches come in all shapes, sizes, and levels of exposure. They can range from a couple of log files unintentionally left available to the public to the leak of hundreds of thousands of users’ personally identifiable information (PII). Don’t think that just because you have a secure network, a leak can’t happen to you.

Let’s start with the happy ending — after a long search, we managed to identify a Netty memory leak in one of our log listeners and were able to troubleshoot and fix the issue on time before the service crashed.