Centralized log collection has become a necessity for many organizations. Much of the data we need to run our operations and secure our environments comes from the logs generated by our devices and applications. Centralizing these logs creates a large repository of data that we can query to enable various types of analysis. The most common types are conditional analysis and trend analysis. They both have their place, but trend analysis is perhaps the more often underutilized source of information.

Production logs can help ensure application security, reveal business insights and find and understand errors, crashes, and exceptions. But as useful as logs are, they’re difficult to manage and hard to keep track of. Making matters worse is that as log data volume grows, so does the difficult task of maintaining and managing them. It’s for this reason that developers, DevOps engineers, and CTOs turn to log management tools.

We cover a lot of ground this week with posts on general monitoring principles, home automation, how CERN uses open source projects in their particle acceleration work, and more.

What is whitebox monitoring? Why do we monitor our systems? What is the Azure Monitor plugin and how can I use it to monitor my Azure resources? Recently, I spoke at Swetugg 2018, a .NET conference held in Stockholm, Sweden to answer these questions. In this video you’ll learn some basic monitoring principles, some of the tools we use to monitor our systems, and get an inside look at the new Azure Monitor plugin for Grafana.

Central storage is vitally important in log management. Just as storing and processing logs into lumber is done in one place, a sawmill, a central repository makes it cheaper and more efficient to process event logs in one location. Moving between multiple locations to process logs can decrease performance. To continue the analogy, once boards are cut at a sawmill, a tool such as a wood jointer smoothes out the rough edges of the boards and readies them for use in making beautiful things.

We are happy to inform our users that a new Docker logging plugin is available on the Docker Store! Using this plugin, users can easily ship container logs directly to Logz.io, and enjoy the following benefits.

Serverless apps are growing in popularity, thanks to tools like AWS API Gateway and Lambda, and a growing number of powerful frameworks that simplify development and deployment. Complex applications are still complex, however, and regardless of your platform you’ll still need to think about observability.