This is going to be the first of at least two posts about how we, in the Mezmo platform team, use Pipeline to handle metrics. It’s an ongoing effort and everything might change as we move forward and learn, but this is our plan and vision.

In our webinar, Mastering Telemetry Pipelines: A DevOps Lifecycle Approach to Data Management, hosted by Mezmo’s Bill Balnave, VP of Technical Services, and Bill Meyer, Principal Solutions Engineer, we showcased a unique data-engineering approach to telemetry data management that comprises three phases: Understand, Optimize, and Respond.

Imagine a well-designed plumbing system with pipes carrying water from a well, a reservoir, and an underground storage tank to various rooms in your house. It will have valves, pumps, and filters to ensure the water is of good quality and is supplied with adequate pressure. It will also have pressure gauges installed at some key points to monitor whether the system is functioning efficiently. From time to time, you will check pressure, water purity, and if there are any issues across the system.

I recently attended SRECon in San Francisco on March 18 - 20, a show dedicated to a gathering of engineers who care deeply about site reliability, systems engineering, and working with complex distributed systems at scale. While there were a lot of talks, I’ll focus on a few areas that gave me the most insight into how having the right data impacts an SREs and an organization’s success.

In our recent webinar hosted by Bill Balnave, VP of Technical Services, and Brandon Shelton, our Solution Architect, we discussed how data's continuous growth and dynamic nature cause DevOps and security teams to lose confidence in their data. The uncertainty about the content of telemetry data, concerns about its completeness, and worries about sending sensitive PII information in data streams reduce trust in the collected and distributed data.