Welcome to another monthly update on what’s new from Sysdig! This month’s big announcement is our new support for Prometheus as a managed service. There are several individual features behind this which we cover in more detail below, but here is a summary: Also, Kubernetes 1.22 was released and we shared our review of what to look out for. Go check out our Kubernetes 1.22 – What’s new? post if you haven’t already.

Welcome to the Civo update for August 2021. It's was a busy month, with the big news being the launch of Civo Academy: A full Kubernetes learning program consisting of over 50 videos created in-house by the team here at Civo. We also kicked off the Civo DevOps Bootcamp! The first few live stream installments have been a huge success, helping developers at any stage of their career learn more about DevOps fundamentals.

It has been a while since I have been excited to write about encrypted tunnels. It might be the sheer pain of troubleshooting old technologies, or countless hours of falling down the rabbit hole of a project’s source code, that always motivated me to pursue a better alternative (without much luck). However, I believe luck is finally on my side.

to unleash the power of cloud native. Anyone in technology knows that one of the keys to success is GREAT technology partnerships. As technology becomes more complex, we look to technology partnerships to help drive digital transformation through growth and innovation.

This month, we have many exciting announcements to make and features to demo like Codespaces with Qovery, deploy on AWS, organization settings, billing, member management, etc. You do not want to miss it!

If you haven’t had a chance to check out VMware Tanzu Mission Control, you’re missing out on one of the greatest tools available to manage Kubernetes. However, it’s not just for managing one cluster; rather, it delivers “fleet-wide” management with a focus on policy. Policy management is powerful because it enables us to, for example, limit access to certain users or prohibit pulls from specific container registries.

Time is invaluable. Besides being one of those can’t-argue-with universal truths, this is also one of the guiding principles behind Komodor; the promise behind our ‘troubleshooting efficiently and independently’ motto. ‘Pods Status and Logs’ is the latest of our timesaving features that enables you to quickly drill down in the pods of an unhealthy service, all from the comfort of your Komodor dashboard.

We recently attended the 2021 Cloud-Native Days Summit, where our co-founding CEO Ben Ofiri gave a lightning round talk on How to Troubleshoot Kubernetes With Confidence. In case you missed it, here’s a recording and transcript for your convenience.

Kubernetes and containerization of applications brings many benefits to software development, enabling speed, agility, and flexibility. The maturation of the Kubernetes ecosystem accelerated quickly in the last few years, leaving users with a multitude of choices when it comes to Kubernetes tooling and services. The major cloud providers (AWS, Azure, and Google Cloud) have introduced services specifically to help users run their Kubernetes applications more efficiently and effectively.

Extended Berkeley Packet Filter (eBPF) is a relatively new feature for Linux kernels that has many DevOps, SREs, and engineers excited. But is it a one-stop shop solution for all of your Linux kernel needs? Let’s take a look at what eBPF does well, and how it stacks up against standard Linux iptables.

The idea to fully manage applications and infrastructure using a Git-based workflow, or GitOps, is gaining a lot of traction recently. We are seeing more and more Shipa users adopting GitOps as the cloud-native deployment methodology. While it is no secret that ArgoCD and FluxCD are by far the most used tools today, we see FluxCD users trying to address the challenges below.

How can you run a fully managed Kubernetes in a private cloud at half the cost of Amazon EKS (Elastic Kubernetes Service)?

A fundamental element of the Kubernetes microservices system is the services model, which gives teams greater understanding of how their applications are deployed. These objects running within pods and containers, by extension, are RESTful since they’re based on APIs. However, DevOps teams can’t hope to run a tight ship without managing their services. Communication and visibility are absolutely crucial in a Kubernetes system.

Kubernetes operations (kOps) is one of the official Kubernetes (K8s) projects. The kOps project allows for rapid deployment of production-grade K8s clusters in multiple cloud platforms. By leveraging yaml manifests, kOps delivers a familiar experience to users who have worked with kubectl. Similar to K8s clusters in popular cloud platforms, kOps helps set up self-managed clusters to easily deliver high availability.

Companies today are adopting software as a service (SaaS) at a rapid pace. There are many factors contributing to this trend, including: Many large enterprises, particularly in banking and financial services, have been hesitant to adopt SaaS because it challenges existing risk management models already in place for software.

I worked at Google for six years. Internally, you have no choice — you must use Kubernetes if you are deploying microservices and containers (it’s actually not called Kubernetes inside of Google, it’s called Borg). But what was once solely an internal project at Google has since been open-sourced, and has become one of the most talked-about technologies in software development and operations.

Shipa is designed to make it simple for developers to run their code on Kubernetes without having to know Kubernetes and for platform engineers to enforce controls and policies.

Are you looking to benefit from automation but lack the experience to leverage an API? To equip you with the tools you need to start utilizing APIs and automation, we’ve put together these helpful Beginner FAQs covering common terminology, methods, and tools for testing APIs.

Spot by NetApp allows its users to manage their application infrastructure using a variety of provisioning tools. One of these tools is Terraform, an infrastructure as code (IaC) tool that allows users to build, change, and version infrastructure safely and efficiently. Spot by NetApp solutions supports multiple Terraform resources, such as Elastigroup, EMR Mr scaler, Managed Instance, and Ocean clusters for different cloud providers, and many more.

In the Kubernetes ecosystem there are a variety of ways for you to provision your cluster, and which one you choose generally depends on how well it integrates with your existing knowledge or your organization’s established tools. Kubespray is a tool built using Ansible playbooks, inventories, and variable files—and also includes supplemental tooling such as Terraform examples for provisioning infrastructure.

Modern, cloud-based software development lifecycles have quickly evolved from waterfall and are fully embracing the agile principles of DevOps. As part of this shift, continuous delivery practices have been adopted, giving organizations the capability to deliver and release code faster and more frequently than ever before. CI/CD tools bring velocity — code is always ready to be deployed, enabling organizations to commit multiple times a day.

The past year has been significant for continued development of both DevOps practices and new developments across the open source community. To that end, Logz.io is moving forward with renewed support for the Fluentd log shipper. This new proxy will serve as an alternative to Filebeat and Logstash, which recently moved away from open source licensing. Additionally, this integration utilizes an HTTP proxy instead of the SOCKS5 proxy necessary for Filebeat.

12 years ago, when I started to work as a Developer, I would not have bet that I will write software and manage infrastructure right from my browser one day. At the time, web IDE did not exist. And managing infrastructure from the browser was super early. Today, it is all possible, and the future looks bright! I am proud to announce that you can edit your code via GitHub Codespaces right from Qovery. Take a look at this short introduction video. Happy coding!

Monitoring services and applications in just a few clicks – that’s the dream of every SRE and developer, but this is very difficult because most applications don’t expose metrics in a standardized format. This article will introduce the current pros and cons of the Prometheus exporters ecosystem and how we leveraged the power of the open-source exporters in Sysdig to radically simplify the user experience to allow you to monitor your applications in just a few clicks.

Last year we cooked a holiday ham using Sysdig. Honestly, just revisiting that makes me hungry, but it got me thinking. What about dessert? Today, I’d like to discuss baking a pie and eating it with Prometheus Remote Write. But not just any pie: a Raspberry Pi. Specifically, I’d like to introduce you to Pi-hole, an open-source project that has become very popular in the community. In this article you’ll learn how easy is Monitoring Pi-hole with Prometheus Remote Write.

We are excited to announce that Prometheus Remote Write functionality is now generally available in Sysdig Monitor. This feature allows Prometheus users to easily push metrics directly from their Prometheus servers to Sysdig’s Managed Prometheus Service. Sysdig Monitor provides not only a scalable long term storage solution for custom metrics but also radically simplifies Prometheus monitoring.

Qovery is a continuous deployment platform. Users deploy apps of all kinds, written in any language and framework they choose. The freedom users have come with a cost for the Qovery core team - the broad scope Qovery has to cover, makes it harder to make the deployment process stable and straightforward for everybody. It's easy to create a service focused on just one language or framework - supporting all of them requires considering many more factors.

Kubernetes is the de-facto platform for orchestrating containerized workloads and microservices, which are the building blocks of cloud-native applications. Kubernetes workloads are highly dynamic, ephemeral, and are deployed on a distributed and agile infrastructure. Although the benefits of cloud-native applications managed by Kubernetes are plenty, Kubernetes presents a new set of observability challenges in cloud-native applications. Let’s consider some observability challenges.

On August 11th, 2021, we conducted a workshop to discuss implementing a GitOps workflow using ArgoCD, Crossplane, and Shipa. We were overwhelmed by the hundreds of people who registered for the session. We believe it shows a few important challenges have to be solved for GitOps to become mainstream across different enterprises. While GitOps brings many benefits, we often see teams facing challenges.

Lately, enterprises are moving towards a hybrid solution that offers the best of both worlds. A hybrid cloud setup combines two infrastructures like a private cloud with one or more public cloud further enabling communication between each distinct service. To maximize returns, a hybrid cloud strategy equips the enterprise with greater flexibility and control by moving workloads between clouds as costs and resources fluctuate.

We are excited to announce a new global release of our software with unique API visibility features to help organizations discover problems with their cloud services well before they impact customers in production.

When you’re troubleshooting an application on Google Kubernetes Engine (GKE), the more context that you have on the issue, the faster you can resolve it. For example, did the pod exceed it’s memory allocation? Was there a permissions error reserving the storage volume? Did a rogue regex in the app pin the CPU? All of these questions require developers and operators to build a lot of troubleshooting context.

Komodor is a Kubernetes-native platform we’ve created to streamline troubleshooting. It was born out of frustrations we felt as developers, when we were required to waste hours of our time on troubleshooting, instead of focusing on what we really wanted to do - creating and innovating. Komodor sits on top of your K8s cluster and integrates with every existing tool you have, be it CI/CD, repo, monitoring, alerting, or communication.

Can you run HAProxy as a Docker container? Yes! Did you even need to ask? Docker is ubiquitous these days and you’ll find that many applications have been Docker-ized; the HAProxy load balancer is no exception. Pardon the cliché, but HAProxy was born for this. As a standalone service that runs on Linux, porting it to Docker certainly seemed natural. Why would you want to run your load balancer inside of a Docker container? Are their performance penalties when doing so?

Containers are lightweight, portable, easily scalable, and enable you to run multiple workloads on the same host efficiently, particularly when using an orchestration platform like Kubernetes or Amazon ECS. But containers also introduce monitoring challenges. Containerized environments may comprise vast webs of distributed endpoints and dependencies that rely on complex network communication.

Spot by NetApp serves hundreds of customers across industries, with different systems, environments, processes and tools. With this in mind, Spot aims to develop our products with flexibility so that whatever the use case, companies can get the full benefits of the cloud. Spot easily plugs into many tools that DevOps teams are already using, from CI/CD to infrastructure as code, including Terraform.

Spot Ocean offers best-in-class container-driven autoscaling that continuously monitors your environment, reacting to and remedying any infrastructure gap between the desired and actual running containers. The way this typically plays out is that when there are more containers than underlying cloud infrastructure, Ocean immediately starts provisioning additional nodes to the cluster so the container’s infrastructure requirements will be satisfied.

In a never-ending game of cat and mouse, threat actors are exploiting, controlling and maintaining persistent access in compromised cloud infrastructure. While cloud practitioners are armed with best-in-class knowledge, support, and security practices, it is statistically impossible to have a common security posture for all cloud instances worldwide. Attackers know this, and use it to their advantage. By applying evolved tactics, techniques and procedures (TTPs), attackers are exploiting edge cases.

We’re excited to announce Calico v3.20! Thank you to everyone who contributed to this release! For detailed release notes, please go here. Below are some highlights from the release.

Docker containers have taken the software industry by storm. Ever since its launch in 2013, Docker’s usage and popularity have grown at a rapid pace. Docker has saved organizations from the challenges of managing dependency and version conflicts across multiple environments by providing a portable, secure, and (most importantly) reliable container technology for shipping applications.

Minimizing and automating the path from development and production is necessary in order to stay competitive and keep customers happy. As engineering teams strive to solve this by quickly and efficiently rolling out new features, updates, and bug fixes, continuous integration and deployment (CI/CD) has come to be regarded as an industry best practice. One of the most popular CI/CD solutions is Jenkins, an open-source job execution system.

No matter what you’re using Kubernetes for, visibility into your applications’ performance and activity is a beneficial and often essential undertaking – essential, but colossal, requiring entire teams dedicated to nothing but maintaining deployments, auditing, debugging, and keeping up with compliance. Kubernetes has robust support documentation dedicated exclusively to assisting customers with Monitoring, Logging, and Debugging.

Due to a great synergy between our products, I am happy to announce that Cloudflare and appfleet are joining forces! The appfleet platform is shutting down, with all clusters going offline on October 31st 2021.

In the daily life of a Site Reliability Engineer, the main goal is to reduce all the work we call toil. But what is toil? Toil is the kind of work tied to running a production service that tends to be manual, repetitive, automatable, tactical, devoid of enduring value, and scales linearly as a service grows. This blog post describes our journey to automate our nodes rotation process when we have a new AMI release and the open source tools we built on this.

Virtualization and cloud have forced the need for automation. In the “old” days, it would take weeks for a new physical server to arrive. There was little pressure to install and configure the operating system on it rapidly. We would insert a disc into the drive and then follow our checklist. A few days later, it would be ready to use. But the ability to spin up new virtual machines (VMs) in minutes required us to get better at automating this process.

Virtual machines (VMs) have transformed infrastructure deployment and management. VMs are so ubiquitous that I can’t think of a single instance where I deployed production code to a bare metal server in my many years as a professional software engineer. VMs provide secure, isolated environments hosting your choice of operating system while sharing the resources of the underlying server. This allows resources to be allocated more efficiently, reducing the cost of over-provisioned hardware.

We're excited to announce Civo Academy: A full Kubernetes learning program consisting of over 50 videos created in-house by the team here at Civo.

In Episode 1 of the OCTOpod, Alan Clark talks with Thierry Carrez about open source communities: what they are, how they work and how you can get involved.