Operations | Monitoring | ITSM | DevOps | Cloud

This article is part of a series on setting up an end-to-end monitoring and alerting stack using Prometheus. Continuing our series on setting Prometheus in a Docker container, we will add a Grafana instance to our Prometheus setup. Please refer to the previous article where we use docker compose to run Prometheus and Alertmanager together as that forms the basis to run multiple related containers. We will add a container to run Grafana to the same compose file in this article.

This blog post will uncover how ilert status pages work, the challenges we encountered while developing this feature, and the problem-solving approaches we adopted.

In January 2025, The Network & Information Security Directive (NIS2) goes into effect. This Directive requires organizations across the EU and those that serve EU customers to strengthen their cybersecurity posture.

In today’s digital world, IT is not a ‘nice-to-have’ but the backbone of every company. Streamlined IT operations are therefore essential for success and even survival. However, technical faults and failures are unavoidable. This is where IT alerting comes into play – a crucial component of IT service management that helps to identify and resolve problems quickly.

Today, every enterprise is digital. Regardless of industry, every business must incorporate digital technologies and strategies into its operations to remain competitive. Maintaining reliable IT infrastructures and digital services while minimizing downtime due to unplanned outages is critical to business success.

One of the most frequent questions I receive from customers is, “What are the best practices to represent my services in PagerDuty?” This question is not easy to answer, but there is a general consensus that the representation needs to be both accurate and visually appealing. This idea got me thinking about our many customers in the beauty and fashion industry.

We’re thrilled to announce the launch of OnPage’s new Multiple Account Login feature. Designed to simplify critical communication workflows and safeguard data security for users working across multiple organizations, this functionality allows them to switch effortlessly between OnPage accounts without the need for repeated logins. Each OnPage account remains securely independent, ensuring that communication is organization-specific and private.

At FireHydrant, we know that alert management is about more than just getting notifications to the right people — it’s about reducing stress and fatigue, balancing workloads, and empowering your team to respond with confidence. That’s why we’re excited to unveil Round Robin for Signals Escalation Policies, a feature designed to make alert escalations smarter, fairer, and more team-friendly by allowing you to automate the sequential assignment of new alerts.

Event-driven automation is a powerful approach to managing enterprise IT environments, allowing systems to automatically react to enterprise events (Observability / Monitoring / Security / Social / Machine) and reducing or removing the need for manual intervention. This post discusses 11 common automation tasks that are ideal for enterprise DevOps teams looking to enhance operational efficiency, reduce downtime, and ensure business continuity. Struggling with ideas for where to start?

More than ever, DevOps teams are constantly tasked with improving collaboration, accelerating software development, and ensuring smooth operations. However, traditional monitoring and alerting methods, often called a “black box approach,” offer limited insight into system performance. As a result, teams rely on reactive approaches, only responding to incidents after they occur without prior planning or strategy.

A status page forms a key part of your incident communication strategy. When it comes to setting up a status page, you have two options: We will examine the pros and cons of each option along these dimensions: For 1, if you choose a self-managed, open-source or custom solution, it's in your control. For a managed solution, you are limited by the provider's feature set. For 2, if you choose a self-managed solution, your team is responsible for the quality of the service.

I want to walk you through how incident management has evolved, drawing from real data and the experiences of some of the most sophisticated tech organizations out there. I'll also introduce you to a framework we’ve developed at incident.io: the Incident Maturity Model. This framework is the result of thousands of conversations with companies and provides a clear roadmap to help your organization improve its incident management practices—no matter where you're starting from.

My name is Tim Gühnemann, and as an AI engineering working student at ilert, I had the privilege of developing and continuous improving ilert AI, ensuring it meets the needs of our customers and aligns with our vision. ‍ Our goal was to provide all our customers with access to ilert AI. We aimed to develop a solution that could adapt dynamically and function independently based on our use cases, similar to the OpenAI Assistant API.

If you manage applications running on cloud platforms, you likely depend on multiple cloud vendors and services. These could be infrastructure providers like AWS, GCP or Azure. A vulnerability in any of these services could potentially impact your applications and your users. A cloud platform has many moving parts, many of which are dependent on other third-party providers.

On December 11, 2024, millions of users around the globe experienced disruptions across Meta’s core platforms: Facebook, Instagram, and WhatsApp. Reports of connectivity issues and outages began flooding social media and third-party monitoring platforms as users scrambled to understand what was happening. While Meta issued a statement later in the evening attributing the outage to unspecified “technical issues,” the delayed acknowledgment left countless businesses and users in the dark.

At ilert, one of the key tools in our debugging process is the Event Explorer, which provides an extensive overview of incoming events and their processing lifecycle. By reflecting the event process of an alert source, the Event Explorer allows our team to trace event paths, correlate related data, and identify issues quickly.

Are your IT systems underperforming due to incomplete or outdated data? In ITOps, where quick and accurate decision-making is critical, raw data alone can limit efficiency. Data enrichment adds the context needed to turn basic data into a powerful source of actionable insights.

When we launched our fully-featured Microsoft Teams integration in May, our goal was clear: to provide enterprise teams with the robust and comprehensive toolset they need to manage incidents faster and more effectively – right where they work. It’s all part of our commitment to building the leading enterprise incident management solution. Today, we’ve enhanced our Teams integration by adding the ability to automatically create Microsoft Teams group chats directly from your Runbooks.

We are excited to announce a native integration between ilert and Honeybadger.

We're thrilled to announce a native integration with ilert, combining Honeybadger's full-stack application monitoring with ilert's real-time alert routing and on-call management platform. ilert handles alert routing, escalations, and on-call scheduling, ensuring critical issues always reach the right person at the right time.

By Debbie O’Brien, Chief Communications Officer and Vice President of Global Social Impact at PagerDuty In today’s digitally-connected world, IT outages can be inconvenient at best and extremely challenging at worst.

Today, we’re excited to announce the release of our new ServiceNow integration in beta — designed to give engineers even more control to manage and automate incidents in FireHydrant while seamlessly keeping the rest of the organization aligned in ServiceNow.

Whether it’s your first or hundredth home call shift, preparing yourself both physically and mentally is crucial. These shifts can be unpredictable, demanding, and emotionally taxing, making it essential to prioritize your well being while maintaining your readiness to provide the best possible patient care. By adopting effective time management, organization, and healthy strategies, you can confidently navigate the unique challenges of home call shifts. Key Takeaways (TL;DR)

With increasing numbers of alerts, keeping focus on the important and most critical alerts proves to be more and more of a challenge. A reduction of alert noise, meaning the prevention of too many created alerts and any kind of user notifications, is needed to ensure efficient alert response. While a detailed explanation of this topic is given in this blog post, a flexible and automated setup for your relevant resources can be achieved with Terraform using the ilert Terraform provider.

Mean time to repair (MTTR), sometimes referred to as mean time to resolution, is a popular DevOps and site reliability engineering (SRE) team metric. MTTR identifies the overall availability and disaster recovery aspects of your IT assets or application workloads. The acronym MTTR can cause some confusion since it has different meanings across different industries. Sometimes, MTTR refers to mean time to respond: the amount of time needed to react to a problem.

In our December update, we have significantly revamped and improved manual alerting. If you need to carefully evaluate incidents before distributing them manually to the respective teams or want to send critical operational updates to relevant personnel, you’ll love the new features we’ve introduced! Additionally, we’ve added intelligent filtering options for automatically incoming events.

In the dynamic world of IT operations and software development, downtime or service disruptions can be costly. As businesses rely more on digital infrastructure, managing and responding to incidents effectively is no longer optional—it’s a critical necessity. However, many organizations struggle to differentiate between incident response and incident management, often using the terms interchangeably.

Managing modern IT environments is becoming more complex and fragmented as organizations rely on a broader range of applications and services, including cloud, hybrid infrastructure, microservices, and legacy systems. This complexity and velocity surpass human capacity and old processes, making it challenging for IT teams to respond efficiently to incidents.

The handoff between IT operations (ITOps) and incident management is often chaotic. NOC operators receive an overwhelming deluge of noisy low-priority alerts, which prevents them from detecting actionable, important alerts. This delay causes tickets to pile up, SLAs breached, and unnecessary assignments and escalations to L2 and L3 engineers. Concurrently, L1 analysts react to user-initiated tickets with little to zero context, forcing them to escalate the issues.

At FireHydrant, we believe that what happens after incidents is just as important as what happens during – and that’s why Retrospectives have always been a cornerstone of our product. Today, we’re proud to introduce the most powerful, customizable, and collaborative retrospective experience you’ll find anywhere.

Observability refers to the ability of the DevOps team to track, monitor, and measure the state of their pipeline and operations. Without observability, you are working in the dark, unaware of what is working. With the growing complexity of modern IT systems, DevOps observability is no longer optional. Gartner estimates that by 2026, 50% of enterprises implementing distributed data architectures will have adopted data observability tools, up from less than 20% in 2024.

Incident management is all about efficiently handling and resolving disruptions in IT services or business operations. It involves spotting, analyzing, and fixing any event that interrupts or could potentially disrupt critical services. The goal is to minimize downtime, keep service quality high, and ensure business continuity. This process includes documenting everything for future reference and improvement, helping organizations learn from past incidents and develop better response strategies.

There are plenty of good SRE/Ops related podcasts out there. I follow a few of them and listen to episodes whose titles sound interesting. The problem with podcasts is that some episodes focus on one topic, and other episodes deal with a host of topics. In between there is filler and things that are not relevant to the topic but are necessary to carry on a conversation. Spending 30-60 minutes listening to podcasts is not always a great use of time.

When things go wrong with your software systems, you need a reliable way to alert the right people and manage incidents. To help you make the best decision, we have summarized the G2 reviews of some of the most popular on-call management tools.

StatusGator continues to demonstrate its value by providing early warning alerts for service disruptions, often detecting issues before official acknowledgment. Below, we highlight key incidents from November 2024 where StatusGator’s monitoring helped users stay ahead.

In the fast-paced world of IT, having a reliable IT alerting software is crucial to ensure swift issue resolution and minimal downtime. The right IT alerting software not only notifies you of critical incidents but also ensures that your team is equipped with tools to respond promptly and effectively. For 2025, we’ve evaluated the top IT alerting software based on features, usability, and a strong focus on mobile app capabilities.

Setting Up Incident Management in Slack Incident Response and Resolution Building a Custom Slack Incident Bot Roles and Responsibilities Optimizing Your Incident Management Process.

On August 28th, 2023—right in the middle of a UK public holiday—an issue with the UK’s air traffic control systems caused chaos across the country. The culprit? An entirely valid flight plan that hit an edge case in the processing software, partly because it contained a pair of duplicate airport codes.

In a DevOps setting, incident management is all about quickly identifying, analyzing, and fixing issues that disrupt IT services. Unlike traditional IT Service Management (ITSM), which often works in isolated teams, DevOps encourages collaboration between development, operations, and business teams. This teamwork ensures that when problems like server outages or software bugs occur, they are handled swiftly and effectively. DevOps incident management is all about being agile and flexible.

Continuing our series on setting up Prometheus in a container, this article provides a step-by-step guide for how to configure alerts in Prometheus. We will add alerting rules and deploy Prometheus Alertmanager with Slack integration. If you follow the steps in this article, you will end up with a containerized setup for: Let's get started.

At AWS re:Invent 2024, PagerDuty is strengthening its long-standing partnership with Amazon Web Services (AWS). Together, we’re launching new AI and automation tools to enhance operational efficiency and help teams deliver superior customer experiences. With a plugin for Amazon Q, and integrations with Amazon Bedrock and Amazon Bedrock Guardrails, PagerDuty Advance is redefining what it means to respond to incidents faster and smarter.

On-call rotation is a system where team members take turns being available to handle urgent issues outside regular working hours. This is crucial in fields like IT, healthcare, and customer service, where quick responses can greatly affect service continuity and customer satisfaction. The on-call engineer is tasked with diagnosing and fixing problems to minimize disruptions and maintain platform stability.

On-call rotations are crucial for ensuring that technical teams are ready to tackle incidents, outages, or emergencies outside of regular hours. (Check our detailed guide on understanding on-call rotations in incident management). This system assigns specific team members to be available for immediate response, ensuring someone is always on duty to address critical issues.