Operations | Monitoring | ITSM | DevOps | Cloud

Having quick access to metrics and health signals from your AWS environment is paramount to identifying issues expediently and monitoring the effects of any deployed fixes. Datadog is proud to partner with AWS for the launch of CloudWatch Metric Streams, a new feature that allows AWS users to forward metrics from key AWS services to different endpoints, including Datadog, via Amazon Kinesis Data Firehose with low latency.

In this latest consideration guide created to assist IT operators in which tools they choose for a range of different processes and activities, we’ve gathered a range of insights from some of our favourite specialists to take you through the essentials tools of configuration management. Configuration management is a systems engineering practise that ensures consistency of the configuration supporting both software and hardware of a company or product.

Advertising tycoon David Ogilvy famously remarked, “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.” Replace the word “half” with “one-third” and “advertising” with “public cloud” and you’d describe what enterprises are grappling with right now. They know that not all of the cloud resources they’re paying for are being used, but they don’t know which ones those are.

At Cloudsmith, we strive hard to ensure that private Cloudsmith repositories work with any build and release process that our customers use. Our mission is to be the universal package management solution that any modern DevOps workflow requires.

While containers are known for their multiple benefits for the enterprise, one should be aware of the complexity they carry, especially in large scale production environments. Having to deploy, reboot, upgrade or apply patches to patches to hundreds and hundreds of containers is no easy feat, even for experienced IT teams. Different types of Kubernetes solutions have emerged to address this issue.

Your business seeks a high-performance, flexible, cost-effective, and more secure alternative to Multiprotocol Label Switching (MPLS). Welcome to software-defined wide area networks (SD-WANs)! An improvement over legacy dedicated MPLS circuits, SD-WAN solutions deliver easily manageable networks with a high quality of service (QoS). Yet, your IT team tells you that SD-WANs aren’t cloud-optimized or mobile-device friendly.

Speedscale ‘SpeedChat’ Episode 2: Shift-Left vs. Shift-Right Throwdown featuring Nate Lee (Founder, Speedscale), Ken Ahrens (Founder, Speedscale) and Jason English (Principal Analyst, Intellyx).

Topology is a multilayered map showing how everything in the IT environment is related. It's similar to Google Maps, which gives you a bird's eye view into an area and how everything is interconnected. Also, in Google Maps, you can see how traffic is flowing and which intersections may be causing bottlenecks. A view into topology allows similar visibility. You can see how components of an IT system are laid out to interact with each other.

If memory serves (it was over 20 years ago), that’s what the free t-shirt that Microsoft gave me for being an official Windows 98 beta tester said. And yes, being a beta volunteer required me to haul my desktop PC to Redmond, WA (from Seattle)…because…I guess that’s how it was done back then? I remember they gave us pizza, and I needed to install more RAM. As I said before, it was a while ago; however, the whole “change is inevitable” thing is still valid today.

Kubernetes orchestrates clusters of machines to run container-based workloads. Building on the success of the container-based development model, it provides the tools to operate containers reliably at scale. The container-based development methodology is popular outside just the realm of open source and Linux though.

Now that you are starting to build your FinOps team and slowly changing your organizational culture, it’s time to start putting the teams to work - TOGETHER! In this blog I will provide you with some tools and processes to get you started.

CircleCI has released new Arm-based compute in preview. Arm processors and architectures are becoming widely available as development teams adopt them as compute nodes in many application infrastructures. Organizations that need to run microservices, application servers, databases, and other workloads in a cost-effective way will continue to turn to the Arm architecture.

Patching vulnerabilities can be daunting, especially if the information is confusing, and the process to deploy patches is long. There are going to be times where a zero day presents itself, and the vulnerability requires immediate action. Whether it’s a simple fix or not, the seriousness of some vulnerabilities can put organizations at risk. Deploying patches, especially for critical infrastructure, takes planning.

This hands-on Flux tutorial explores how Flux can be used at the end of your continuous integration pipeline to deploy your applications to Kubernetes clusters.

We're very proud to announce that we have been accepted by the Cloud Native Computing Foundation (CNCF) as a conformant Certified Kubernetes provider for our v1.20 Civo Kubernetes product based on K3s. Every small business starts with a goal of competing with the big fish of the industry, and a huge part of that is having the certification to prove you're providing a compatible service. We're now in the company of some really inspirational organisations...

Fastly is an edge cloud platform that includes a content delivery network (CDN), as well as services for image optimization, video streaming, cloud security, and load balancing. These services are supported by a network of caches in different locations, which enables enterprise-scale companies to deliver applications to users as quickly as possible, even in times of peak traffic.

Apache Cordova, since its release in 2009, has created a paradigm shift for mobile application development. Before Cordova, only developers who knew platform dedicated languages for a particular type of mobile operating system could develop native mobile applications. There was Objective-C for developing iOS apps, or Java for Android apps and platforms like Blackberry. Windows Phone also had languages dedicated to building their mobile applications.

DevOps tool of the month is a series, where each month I will introduce one new useful DevOps tool in 2021 For March I chose: Shipa – Shipa’s cloud native application management framework makes working with Kubernetes for developers extremely easy.

FireHydrant uses a CLI for some developer actions, called fhd (FireHydrant developers). Previously, we might have distributed workflows among new developers by having them copy/paste or clone scripts down to their machines--but Crystal lets us encapsulate shared tooling in a compiled binary. This way, we have a CLI that developers can install quickly, and that works seamlessly with our other tools.

Software development by distributed teams is nothing new. But since 2020, it’s no longer just teams that are globally dispersed, it is the individual team members themselves. Remote working is the new normal. So in this unpredictable, “modern” world we’re in, how do you put together a solution that delivers for every single team member, no matter their location?

Following the Open Operator Collection announcement from November, Canonical is today proud to announce the availability of Juju Operator Lifecycle Manager (OLM) 2.9. This new release of Juju brings new capabilities for Kubernetes operators as well as smooth integration with the Open Operator Collection.

No one can deny the importance of a proper monitoring system for the effective management of IT infrastructure. You need the most efficient solutions and monitoring tools to optimize performance, make the most out of your resources, and be able to deal with errors and failure conditions. While the traditional way of IT monitoring involves the use of reports, it does have certain limitations.

During the next seven weeks, our team will work to improve the overall experience of Qovery. We gathered all your feedback (thank you to our wonderful community 🙏), and we decided to make significant changes to make Qovery a better place to deploy and manage your apps. This series will reveal all the changes and features you will get in the next major release of Qovery. Let's go!

A floor of commotion bustling with people holding phones and shouting out purchase and sell orders, some using hand signals to communicate over the noise. This was a common scene on Wall Street in the 1980s. Nowadays, transactions happen at the push of a button with traders sitting directly in front of a computer. In fact, the computer has made it possible to automate the buying and selling of securities, leading to an era of high-frequency, algorithmic trading.

March 1, 2020, was a Sunday. I was getting ready for a work trip to San Francisco. Just before heading to the Berlin airport, I checked my phone. There was a text from my boss: Hey Lena, trying all channels. Don’t fly. I didn’t fly that day and I haven’t flown since. Two days later, as planned, I went from interim to permanent VP of Product Engineering at CircleCI. At the same time, Europe was going into lockdown, then the world went into lockdown.

Modern applications are interdependent on different devices and servers. If you are not familiar with your software structure, you will encounter problems when you make changes within your application. To prevent this, you need to observe application discovery and dependency mapping (ADDM).

March 25th, 2021 – Canonical, Collabora and Nextcloud announce the immediate availability of a content collaboration platform for 64bit ARM for both consumers and enterprises. Building on the prior Nextcloud Ubuntu Appliance it adds with Collabora Online, the first viable self-hosted web office solution on the popular Raspberry Pi 4 platform.

When starting a cloud migration project, one of the most important and often challenging parts is to have an accurate understanding of what you are trying to migrate. Over time, companies start new projects, which means creating new infrastructure, adding servers, databases, etc. This is a normal part of the development cycle. However, despite best efforts, inventories get out of sync.

The key to maximizing revenues and ensuring foolproof security is to actively monitor infrastructure. With virtualization growing increasingly, virtual machine or VM performance monitoring is one of the most crucial parts of infrastructure monitoring. This is because as opposed to a physical server or machine, detecting problems in a VM can be even more challenging.

Monitoring the current state and performance of applications is critical for IT Ops and DevOps teams alike. Understanding the health of an application is one of the most effective ways of anticipating potential bottlenecks or slowdowns, yet it’s one of the largest challenges faced by many organizations that build and deploy software. This is largely due to applications’ distributed and diversified nature.

“What is GitOps?” – a question which has seen increasing popularity on Google searches and blog posts in the last three years. If you want to know why then read on. Quite simply, the coining of GitOps is credited to one individual, and pretty smart guy, Alexis Richardson. He’s so smart that he’s built a multi-award-winning consultancy, Weaveworks, and a bespoke product, Flux, around the GitOps concept.

Programming is often thought of purely as a problem-solving activity. This may be true for the lone coder in their garage, but in the multi-person environment of an Agile team, such problem solving must be collaborative. In this article, we’ll look at the role of communication in software development, particularly in an Agile framework. Covid-19 has forced an unprecedented shift to remote working so we’ll finish up with a discussion of how Agile can be implemented in a remote setting.

With the new Contexts API release, developers can save their team valuable time while enhancing security practices. We know maintaining your organization’s security is crucial. There is the need to meet strict compliance guidelines, such as FedRAMP and GDPR, and what seems to be an increasing number of breaches, like the compromise of over 150,000 video security cameras as a result of a targeted Jenkins server.

Running an Artificial Intelligence (AI) infrastructure on premise has major challenges like high capex and requires internal expertise. It can provide a lot of benefits for organisations that want to establish an AI strategy. The solution outlined in this post illustrates the power and the utility of the universal Operator Lifecycle Manager (OLM) using Juju, a universal OLM, to create and manage on premise AI infrastructure.

The term cloud native is widely used when thinking about computing and software development, encompassing a wide range of concepts that are regularly used in technology. Let’s break it down and take a closer look – what does cloud native really mean?

Customer Profile One of the top 10 largest financial services companies in the world 200,000+ employees worldwide. Serving tens of millions of customers. With operations in more than 60 countries, the Interlink Incident Alert Management app serves an audience of thousands of service owners and business stakeholders - across 20+ global markets

One of the most powerful aspects of AWS is their Identity and Access Management (IAM) service. The obvious aspect of its power is that it controls who can do what with all the resources inside your AWS account. But the non-obvious side is how configurable it is. You can encode permissions that are so finely grained that a Lambda Function could, for example, be given just enough permissions to be able to read one attribute from one record for the current user of a DynamoDB Table.

It can be really exciting when your development team is growing fast! But then you soon realize that managing all the developer tools to constantly create new projects or add users is becoming a full-time job. Well, not anymore. At least, not for Rollbar. We’re releasing our HashiCorp Terraform Verified Provider for Rollbar today, built in partnership with HashiCorp.

After we’d fixed Aparna’s network issue, James came to see me at my desk. Masks on, socially distanced and all that, but it was nice to have some face-to-face time. James is cool – that dry British humor and not your classic IT Ops dude. He’s been here forever and mentored me when the CIO, Charlie, hired me as the first SRE here a year or so ago. I lucked out really.

Server 3.x, which is now available to all CircleCI customers, was designed to meet the strictest security, compliance, and regulatory restraints. This self-hosted solution can scale under heavy workloads, all within your team’s Kubernetes cluster on your private network, but with the dynamically scaling cloud experience of CircleCI. Offering an exceptional on-premise service means keeping that service up-to-date with the latest releases.

Containers have revolutionized how we distribute applications by allowing replicated test environments, portability, resource efficiency, scalability and unmatched isolation capabilities. While containers help us package applications for easier deployment and updating, we need a set of specialized tools to manage them.

Most applications begin with a small to medium-sized user base. Even with migration projects, you would not immediately open your new application to the entire existing user base. Instead, you would first test with some internal users, then open up to early adopters. Nevertheless, if your application is successful, at some point you will face the need to scale it. Nevertheless, if your application is successful, at some point you will face the need to scale it.

The hype of 5G is ushering in the potential for drastic network change and is driving service providers to prepare for the future. Today’s networks provide ubiquitous all-purpose IP connectivity to support all services.

COVID-19 certainly accelerated some trends. For example, in my view, COVID-19 accelerated the pace and progress of digital transformation by five to 10 years, as companies faced the need to adapt to a post-COVID-19 world, involving permanently the higher adoption of remote work, remote education, and digital touch-points. Technologies requiring less human touch and more digital touch-points will be adopted faster in the aftermath of COVID-19.

Keeping up with customer demand and staying ahead of the competition requires fast releases and efficient code. This is where DevOps comes into play. Where DevOps is the practice, mindset, and culture of combining software development (i.e. Dev) and IT operations (i.e. Ops) to accelerate the development lifecycle and provide continuous delivery without compromising quality.

It’s with immense pride and excitement we can announce that Civo is coming out of beta! When we planned out our beta program, which we called #KUBE100, our first priority was getting real-world feedback from a wide range of developers, all at various stages in their Kubernetes journey. We wanted to create a Kubernetes platform that plugged the gaps we saw in the industry – speed of deployment, ease of use, and a simple billing model.

During the next eight weeks, our team will work to improve the overall experience of Qovery. We gathered all your feedback (thank you to our wonderful community 🙏), and we decided to make significant changes to make Qovery a better place to deploy and manage your apps. This series will reveal all the changes and features you will get in the next major release of Qovery. Let's go! Read the previous article: The Future of Qovery - Week #2.

The transition to virtualized infrastructure and software network functions requires a re-evaluation in the telecom production environments. Network function is not alone. It has always been an important piece in the puzzle but requires a set of non-functional bits to complete the big picture and which brings the lifecycle management of network functions to the discussion.

In almost every case, APIs have changed how modern applications connect to their data. Mobile apps, single-page web apps, IoT devices, integration hooks between software—all of these things rely on APIs to fetch, update, delete, and create data. In fact, one set of APIs might serve as the backbone of a website, mobile app, voice assistant device, and more, meaning one data store owns a treasure trove of information about us, the human users.

Speedscale ‘SpeedChat’ Episode 1: Discussing software product management, ‘dogfooding’ and scaling quickly for product/market fit without breaking things.

It has been eight years since The Phoenix Project was published and a lot has changed since then! I started to think about what we’ve learned in that time. It starts with the theory of constraints. I still see it all the time. Organizations take actions which are merely temporary, putting out fires but not solving for the underlying causes of those fires.

And that was February! A month where a rover showed what perseverance means and a small drone what ingenuity looks like. February will be remembered as the month where two robots landed on Mars, telling us all to “dare mighty things”.

We recently launched the public preview of a new tool that lets anyone dashboard anything – for free. Meet Dashboard Server! You can download it free because we wanted to get our incredible dashboards in the hands of more people. The SCOM community know us well but our dashboards are perfect for so many use cases beyond System Center monitoring. And if you love them as much as we think you will, all we ask in return is that you share that love by spreading the word.

Let’s take a moment and think about security in your organization. Security is often separate from other engineering teams such as development, operations, networking, IT, and so forth. If you narrow down your focus to specifically releasing new software or features and functions in existing software, you’ll find that while development and operations are working together very quickly and efficiently, they’re still vaulting these functions and features over to security.

As we wrap up the first quarter of 2021, we wanted to talk about things we should be doing as part of a cloud native strategy for the remaining 3/4 of the year. Moving from traditional monolithic. architectures to a modern microservices approach has many benefits, but still has the greater majority of us baffled in terms of tapping into its full potential.

If you are a .NET developer, monitoring runtime metrics can help you troubleshoot bugs and detect resource inefficiencies in your applications. With Datadog, you can easily collect, visualize, and alert on key .NET runtime metrics, including exceptions, garbage collection statistics, thread count, and more. We have fully integrated .NET runtime metrics into Datadog APM so that you can easily view them alongside your distributed traces, logs, and other telemetry.

This Logz.io Infrastructure Monitoring tutorial will cover how to get started with our latest product, our new Prometheus-as-a-Service metrics solution that’s based on Prometheus. Engineers monitor metrics to understand CPU and memory utilization for infrastructure, duration and serverless execution, or for network traffic. For more advanced metrics monitoring operations, teams can send custom metrics to monitor signals like the number of active users.

Did you know that CloudWisdom’s Bill Analysis tool shows you not just the services currently monitored by CloudWisdom but all services to deliver an overall view of your AWS cost? And if you’ve set up and configured consolidated billing to link multiple AWS accounts, you can include data from all those accounts in that view. You can even add multiple billing orgs to the same CloudWisdom account.

As corporations grow, the need to organize user data and assets into a hierarchical structure becomes critical to to simplify storage access of those assets. LDAP enables organizations to store, manage, and secure information about the organization, its users, and assets. In this guide, we’ll explain what LDAP is, its uses, and how it works.

Take the 2021 State of DevOps Survey here. This year marks the 10-year anniversary of our industry-defining State of DevOps research. You’d think after 10 years that there wouldn’t be much more to say about DevOps, but practices and technologies continue to evolve, the best keep getting better, and we keep unearthing new topics to research.

Managing IT infrastructure is impossible without proper monitoring solutions and tools. Monitoring requires regular checks on the status, and the best way to gather details would be in the form of reports. Advanced IT monitoring solutions provide an automatic diagnosis of performance and availability issues across your IT network; manual interventions help optimize the process. Manual interventions make sure you don’t miss out on any warning signs before reaching the critical points.

In this article, we are going to look at the architecture of Apache ActiveMQ and how to monitor critical metrics of ActiveMQ using Hosted Prometheus and Hosted Grafana. If you would like to follow the steps in this blog, make sure to sign up for the MetricFire free trial. You can use Graphite and Grafana directly from our platform. MetricFire is a Hosted Graphite, Grafana and Prometheus service, where we do the setup and management of these open-source tools so you don’t have to.

The rapid pace of updates and upgrades to operating systems, software frameworks, libraries, programming language versions – a boon to the future of fast-paced software development, has also come to slightly bite us in the back because of having to manage these very many dependencies with their different versions across different environments.

For many software engineers and developers, using standard libraries or built-in objects is just not enough. To save time and increase efficiency, most developers build on work done by others. Whatever the coding problem, there is likely another programmer who has already created a solution for it. There is usually no need to repeat the problem-solving process. This principle is known as Do not Repeat Yourself or DRY.

An on-call schedule tells you and everyone in the team who will be the first responder when an issue happens in production. The on-call team member is responsible for investigating the issue, either fixing the issue herself or adding other people who can help fix it. Having an on-call schedule is important for building reliable systems because making someone responsible for production issues makes sure that they're not ignored.

Adding alerts across your monitoring tools is taking a proactive approach to reliability. But if there are too many alerts, then it can become counterproductive because team members will start ignoring alerts or remove the alerting altogether. Which is why you need a systematic approach to adding alerts and dealing with them.

One of the advantages that Ubuntu brings to the cloud equation is improving an organization’s ability to run in multiple clouds. Running containers on top of Ubuntu further increases portability. Mist is an open-source multi-cloud management platform that helps teams centrally manage and control their Ubuntu instances across many different cloud environments and/or bare metal. This removes some of the operational and financial barriers to running applications in multiple clouds.

Virtana recently published the results of a new State of Hybrid Cloud survey. One of the findings is that 81% of companies in the study who have started their migration to the public cloud have engaged multiple providers. This result tallies with a recent Gartner survey of public cloud users, in which 81% of those respondents said they are working with two or more providers.

I’ve learned a lot about serverless in my first full year at Stackery after joining from the Ops-focused automation company, Puppet. I’ve learned how to deploy rich CMS-backed web applications in a serverless way, how to cast incantations using CloudFormation’s intrinsic functions, but most of all I’ve learned that there’s a lot to learn in order to be successful in AWS.

Mutual of Enumclaw, a 100-year-old insurance company from Washington, modernized their policy processing system in order to reduce TCO and focus on their software and customers - not infrastructure. However, they realized serverless development still presents many DevOps challenges like environments, workflows, documentation and local development.

Azure Storage is like an all you can eat buffet, except the more you eat, the more you pay! This has provided organisations an almost limitless supply of storage, and as we all know, the more that’s available, the more we’ll use. Azure Storage has changed the way many organisations operate both in terms of availability and service.

Kubernetes can bring a wide collection of advantages to a development organization, but efficiently deploying applications to Kubernetes is something many organizations are still working to perfect. Properly using Kubernetes can significantly improve productivity, empower you to better utilize your cloud spend, and improve application stability and reliability. On the flip side, if you are not properly leveraging Kubernetes, your would-be benefits become drawbacks.

During the next nine weeks, our team will work to improve the overall experience of Qovery. We gathered all your feedback (thank you to our wonderful community 🙏), and we decided to make significant changes to make Qovery a better place to deploy and manage your apps. This series will reveal all the changes and features you will get in the next major release of Qovery. Let's go!

In our previous blog post, Using HAProxy as an API Gateway, Part 2 [Authentication], you learned that when you operate HAProxy as an API gateway, you can restrict access to your APIs to only clients that present a valid OAuth 2 access token. In this post, we take it a step further. You will learn how to leverage tokens to grant some users more access than others and then charge for the service.

VoltDB is an ACID-compliant, in-memory relational database designed to support real-time analytics. VoltDB’s in-memory storage, stored procedures, and shared-nothing architecture make it specifically optimized for quickly processing massive streams of data. This means VoltDB is tailored for use cases like online gaming, telecommunication, and financial applications, which require fast data processing.

One of the biggest challenges that DevOps teams face is how to connect their efforts with the priorities of business leaders. In conversations we’ve had, developers and SREs discussed how they need to show business and engineering leaders that they are investing their time solving the right problems, and how solving these problems lead to overall better business outcomes.

Our team at CircleCI has been hard at work this past year. From rolling out major product additions like CircleCI Insights and private orbs, along with educational resources like our developer hub and State of Software Delivery Report that includes the first CI/CD benchmarks for high-performing engineering teams, it seems like every day there is something new to look forward to. All of this has taken place during unprecedented times for everyone across the globe.

Love 'em or hate 'em, mobile push notifications can be very useful. They are not as intrusive as a phone call and have better information formats and control than text messages. Which is why it can be very frustrating to not get push notifications for your favorite product because it doesn't have a mobile app. In this post, we will see how to get mobile push notifications from any service, even if they don't have a mobile app.

In this episode of ShipTalk, Jim Shilts, Developer Advocate at Shipa and the Founder and President of North American DevOps Group (NADOG), chats with Ravi Lachhman, Evangelist at Harness on the “Shifting Complexities in DevOps.” Jim has been working on solving engineering efficiency problems for over 20 years, working at firms such as Build Forge and Electric Cloud, pre-dating the inception of Hudson/Jenkins.

I’ve got some exciting news! With the recent release of Puppet Comply 1.0.2, we’re providing the ability to assess Windows 2016 servers against DISA STIGs, required security standards for companies that do business with the U.S. Department of Defense. This new capability makes it drastically easier for organizations to assess and demonstrate compliance with DISA STIGs.