Operations | Monitoring | ITSM | DevOps | Cloud

I have always found data collection to be a fascinating area of work at Sumo Logic. Collecting data is a critical first step for all the solutions we develop for our customers. After all, to observe the health and performance of your applications, you must first collect all relevant data. It's also an area that has seen some significant activities by the open-source community over the years, which is completely changing the landscape of observability as we know it.

We all know the importance of cataloging, organizing, and breaking down the data in your logs. That process, parsing, makes information easier to find and simplifies subsequent analysis. Now, with Logz.io’s upgraded self-parsing tool, custom parsing rules, and log organization is easier than ever. What’s important is parsing that data out correctly. The better parsed, the easier to query.

Tucker Callaway is the CEO of LogDNA. He has more than 20 years of experience in enterprise software with an emphasis on developer and DevOps tools. Tucker drives innovation, experimentation, and a culture of collaboration at LogDNA, three ingredients that are essential for the type of growth that we've experienced over the last few years.

Which data sources do DevOps teams need in order to achieve observability? At a high level, that’s an easy question to answer. Concepts like the “three pillars of observability”—logs, metrics, and traces—may come to mind. Or, you may think in terms of techniques like the RED Method or Google’s Golden Signals, which are other popular frameworks for defining which types of data teams should collect for monitoring and observability purposes.

Logging is a fundamental part of software development. While an app is being developed, we rely on logging to confirm our inputs and outputs match our expectations. In production, logging can be an invaluable resource for tracking down bugs or measuring how users interact with the app. We can also consider logs as a sort of time-series value, where a timestamp is associated with a user’s specific action. These logs can be structured, gathered, and analyzed to provide teams with more information.

Many companies in Asia Pacific (APAC) were caught in a digital tailspin when Covid-19 hit, sacrificing security practices in their hurry to adjust to the new reality of remote work. Two years on, hybrid work is still the norm as the pandemic continues and seems to be a new way of life moving forward. Catalyzed by the coronavirus, firms big and small are now adopting cloud technologies as we tread deeper into a new data age.

We are happy to provide an open-source Terraform module enabling automatic delivery of CloudTrail logs from S3 to a Kinesis Data Stream.

A Content Delivery Network (CDN) is a distributed set of servers that are designed to get your web-based content into the hands of your users as fast as possible. CDNs produce CDN logs that can be analyzed, and this information is invaluable. Why? CDNs host servers all over the world and are designed to help you scale your traffic without maxing out your load balancers. A CDN also gives you added protection against many of the most common cyber attacks. This activity needs to be closely monitored.

To quote the UK National Data Strategy: Splunk is an advanced data platform that delivers right-time analytics from diverse data sets and that enables organisations to ask questions of all their data. It can be used to mitigate cyber security risk, improve performance, increase reliability and observe what is happening in the cloud.