Operations | Monitoring | ITSM | DevOps | Cloud

Data teams are adopting more processes and tools that align with software engineering, and from talks at the dbt Coalesce conference in 2023, there’s clearly a big push towards adopting software engineering practices at enterprise scale companies. At the moment, there are a lot of tools in the data space for identifying errors in data pipelines, but no tools for responding to these errors, such as coordinating fixes. This is exactly where an incident management platform makes sense to implement.

The software engineering world has become a place where compute, storage, and availability have become the cornerstones of scale. As an industry and as individuals, we should stop to take a closer look at scaling the most important of all resources… our people. In this post I’ve modeled a team with 6 engineers, 2 Sr, 3 Mid, and 1 Jr. This team is getting 450 “units” of work done ( where a unit is just some measure of throughput ) per interval (2 months).

In the world of modern Kubernetes, things have come a long way from the days of a single cluster handling one app. Now, it's common to see setups that span multiple clusters across different clouds. Initially, managing those clusters was a complicated operation with many moving parts. Using tools such as SUSE Rancher, RedHat OpenShift or AWS EKS, made managing multiple clusters somewhat easier.

We could probably sum up the direction the financial services industry is headed in just two words: digital transformation. It’s the buzzy, all-encompassing focus and end goal of financial services organizations near and far, and it’s the big reason why many organizations are thinking about process automation with a greater sense of urgency.

Today, federal agencies rely extensively on Cloud-based SaaS applications for everything from payment processing and document management, to data security and employee workflow automation. These tools help departments to function very efficiently, but because they are being used for essential government functions, it’s vital that they are safe and secure. For example, personnel at The Pentagon or The Department of Homeland Security can’t just choose any software vendor in the marketplace.