Monitoring systems help DevOps teams detect and solve performance issues faster. With Docker and Kubernetes steadily on the rise, it’s important to get container monitoring and log management right from the start. This is no easy feat. Monitoring Docker containers is very complex. Developing a strategy and building an appropriate monitoring system is not simple at all.

Applications fail. Containers crash. It’s a fact of life that SRE and DevOps teams know all too well. To help navigate life’s hiccups, we’ve previously shared how to debug applications running on Google Kubernetes Engine (GKE). We’ve also updated the GKE dashboard with new easier-to-use troubleshooting flows. Today, we go one step further and show you how you can use these flows to quickly find and resolve issues in your applications and infrastructure.

Google Kubernetes Engine (GKE) is the preferred way to run Kubernetes on Google Cloud as it removes the operational overhead of managing the control plane. Earlier today, Google Cloud announced the general availability of GKE Autopilot, which manages your cluster’s entire infrastructure—both the control plane and worker nodes—so that you can spend more time building your applications.

Today, I’m excited to announce the contribution of the sysdig kernel module, eBPF probe, and libraries to the Cloud Native Computing Foundation. The source code of these components will move into the Falco organization and be hosted in the falcosecurity github repository. These components are at the base of Falco, the CNCF tool for runtime security and de facto standard for threat detection in the cloud.

AI and Machine Learning (ML) are key priorities for enterprises, with a recent survey showing that 72% of CIOs expect to be heavy or moderate users of the technology. Unfortunately, other research has found that the vast majority—87%—of AI projects never make it into production. And even those that do often take 90 days or more to get there. Why this disconnect between intent and outcome? What are the roadblocks to enterprise ML? And what can be done about them?

Welcome to another monthly update on what’s new from Sysdig. Our team continues to work hard to bring great new features to all of our customers, automatically and for free! We hope you all managed to make it through January, and happy Lunar New Year! February welcomes the launch of our always-popular fourth annual Sysdig Container Security and Usage report, which looks at how global Sysdig customers of all sizes and industries are using and securing container environments.

Image vulnerability scanning is a critical first line of defense for security with containers and Kubernetes. Today, Red Hat recognized Sysdig as a certified Red Hat security partner based on our work to standardize on Red Hat’s published security data with Sysdig Secure.

In the Modern era of application development, businesses move towards building highly available, fault-tolerant, zero downtime applications to make the user experience and performance smoother and better. One of the essential steps in that process is containerization and orchestration of an application. A Container Monitoring process is as vital as containerizing your application.

For all the talk of digital transformation, there’s one workflow that tends to hinder release velocity: changes to API routing rules. But while—much to the consternation of enterprise developers everywhere—this process has historically remained stubbornly ticket-based, Spring Cloud Gateway removes this bottleneck. The open source project provides a developer-friendly way to route, secure, and monitor API requests.

As a team we have spent many years troubleshooting performance problems in production systems. Applications have gotten so complex you need a standard methodology to understand performance. Fortunately right now there are a couple of common frameworks we can borrow from: Despite using different acronyms and terms, they fortunately are all different ways of describing the same thing.

With build-or-buy decisions, it often comes down to an all-in-one platform or a mixture of best-of-breed technologies. With open-source technology companies can actually get the best of everything. So, why not roll your own platform based on top-notch technologies? The real question is whether enterprises can afford to. Open-source software is free to use, but teams have to invest quite a bit in selecting, introducing, using, and maintaining these technologies.

Welcome to part 2 of our blog series, where we go through how to forward container logs from Amazon ECS and Fargate to Splunk. In part 1, "Splunking AWS ECS Part 1: Setting Up AWS And Splunk," we focused on understanding what ECS and Fargate are, along with how to get AWS and Splunk ready for log routing to Splunk’s Data-to-Everything platform.

CloudHedge Transform provides the user with the option of modifying the data that goes into the container. It uses the data gathered from the X-Ray, a part of the CloudHedge Discover module that has been performed on the process. Using the Transform Platform, the user can currently: The Edit Dockerfile feature can be used to: And the File Selection feature can be used to.

Let’s face it. Once you have consumed your free credit, AWS costs an arm and a leg. This is the price to pay for high-quality services. But how can you reduce your costs without sacrificing quality? This post will show you how to reduce your bill by up to 60% by combining four built-in features in Qovery. There are three categories of costs on AWS. The “data transfer”, the “compute”, and the “storage” costs.

AI is everywhere. Except in many enterprises. Going from a prototype to production is perilous when it comes to machine learning: most initiatives fail, and for the few models that are ever deployed, it takes many months to do so. While AI has the potential to transform and boost businesses, the reality for many companies is that machine learning only ever drips red ink on the balance sheet.

In this episode of Coffee & Containers, North American DevOps Group‘s Jim Shilts speaks with Shipa‘s Bruno Andrade and Fiserv‘s Ken Owens. The topics covered include.

Admission controllers are a powerful Kubernetes-native feature that helps you define and customize what is allowed to run on your cluster. As watchdogs, they can control what’s going into your cluster. They can manage deployments requesting too many resources, enforce pod security policies, and even block vulnerable images from being deployed. In this article, you’ll learn what admission controllers are in Kubernetes and how their webhooks can be used to implement image scanning.

The latest versions of Konvoy and Kommander are now generally available: the D2iQ Kubernetes Platform (DKP) continues on its relentless roll. DKP is the leading independent Kubernetes platform for enterprise grade production at scale and Konvoy and Kommander are the reason why. You can learn more about Konvoy here, Kommander here, and our general approach here.

Following the GA of Kubernetes 1.19 support in AWS, EKS-optimized Ubuntu images for 1.19 node groups have been released. The ami-id of this image for each region can be found on the official site for Ubuntu EKS images.

Today’s enterprises are pushing forward with their digital transformation initiatives to meet customer and market demand. The latest CNCF survey reports that 91% of companies are running Kubernetes and 81% of those companies are running Kubernetes in production. That’s up from 58% in 2018, and the numbers continue to ramp up quickly. There’s several approaches to how enterprises are thinking about adoption and their deployment and management of Kubernetes. I sat down with Lauren E.

With an increasing number of organizations migrating their applications and workloads to containers, the ability to monitor and track container health and usage is more critical than ever. Many teams are already using the Metricbeat docker module to collect Docker container monitoring data so it can be stored and analyzed in Elasticsearch for further analysis. But what happens when users are using Amazon Elastic Container Service (Amazon ECS)? Can Metricbeat still be used to monitor Amazon ECS? Yes!

As our customers scale and utilize Coralogix for more teams and use cases, we decided to make their lives easier and allow them to set up their Coralogix account using declarative, infrastructure-as-code techniques. In addition to setting up Log Parsing Rules and Alerts through the Coralogix user interface and REST API, Coralogix users are now able to use modern, cloud-native infrastructure provisioning platforms.

We are excited to introduce Calico Cloud, a pay-as-you-go SaaS platform for Kubernetes security and observability. With Calico Cloud, users only pay for services consumed and are billed monthly, getting immediate value without upfront investment.

It’s no secret that Amazon Web Services is a powerhouse Cloud provider, and one of the market pioneers in Cloud operations. They do, after all, power some of the world’s biggest and most modern systems we all use and love today. It’s natural then that they attract a lot of users both big and small to deliver high quality and effective solutions. With growing user demand comes the need for new methods of visibility and intelligence.

In this episode of Coffee & Containers, North American DevOps Group‘s Jim Shilts speaks with Shipa‘s Bruno Andrade, VMware Tanzu‘s Gautham Pallapa, and IBM Cloud‘s Jason McGee McGee.

Thank you for reading my last blog on how to modernize age-old applications using automation. Let’s take a closer look at the available automated tools and explore the insights they extract to speed up app modernization. Assessment and Analysis The automation tool for application assessment should: There are free tools provided by cloud service providers, however, they focus more on infra (VMs and Bare Metal) and don’t focus on applications and databases (aspects mentioned above).

Runtime security for Azure Kubernetes Service (AKS) environments requires putting controls in place to detect unexpected and malicious behavior across your applications, infrastructure, and cloud environment. Runtime threats include things like: Even if you’re taking advantage of tools like container image vulnerability scanning, Kubernetes pod security policies, and Kubernetes network policies with AKS, not every risk will be addressed.

If you are using a CI/CD tool, you likely are already familiar with workflows. Generally, workflows are a set of tasks, activities or processes that happen within a specific order. Within Codefresh, a popular workflow is to trigger Codefresh pipelines from Docker image push events. This moves the workflow forward from Continuous Integration to Continuous Deployment. Images can be promoted from one environment to the other through a variety of ways.

Building containers securely, reliably, and consistently at scale is a daunting task. Yet, it’s an imperative for organizations embracing the rapid delivery of high-quality software. This is the scenario addressed by VMware Tanzu Build Service, which can help any enterprise IT group build and update containers automatically. And it’s flexible enough to slot right into any incumbent CI/CD toolchain.

As Kubernetes adoption continues to grow, Kubernetes audit logs are a critical information source to incorporate in your Kubernetes security strategy. It allows security and DevOps teams to have full visibility into all events happening inside the cluster. The Kubernetes audit logging feature was introduced in Kubernetes 1.11.

Code analysis tools are essential to gain an overview and understanding of the quality of your code. This post is going to cover the following While these tools target similar use cases, they differ in their implementation, ease of use, and documentation just to name a few. This post provides an overview of each tool as well as a detailed comparison to help analyse and decide which tool is best suited for your needs.

Kubernetes applications are increasingly making their way to the edge and embedded computing. Storage will quickly follow as the applications that rely on this edge infrastructure become more advanced and naturally carry more state. According to a study by McKinsey and Company, a “connected car” processes up to 25GB of data per hour.

Within just five years, Kubernetes and containers have redefined how software is deployed. Researchers expect the container market to grow by 30% year over year to become a 5 billion industry by 2022. But what is the reason behind this mass adoption of container technology in the enterprise? Download whitepaper Containers are more resource efficient than virtual machines or other legacy app architectures.

Automation and adoption of cloud (public and/or private) are the two key components for the success of Digital Transformation. Automation has been synonymous with the agility and scalability of applications and infrastructure. DevOps ensures quick release of apps all the way from development to test to the stage to production. Cloud ensures quick allocation of resources (Compute, Memory, Storage, Network, etc.) for the applications.

You may have noticed the VMware Tanzu team talking and writing a lot about container security lately, which is no accident. As DevOps and Kubernetes adoption continue their exponential growth in the enterprise, securing container workloads consistently is among the most difficult challenges associated with that transformation. There is a term we have been seeing—and using—a lot lately that encompasses a new way of looking at container security for Kubernetes: DevSecOps.

When we first started our managed Kubernetes beta, we knew utilising K3s as the Kubernetes distribution of choice was the right move. Not only is it light-weight and quick to deploy, K3s has features ideally suited for the scenarios we envisioned our users would encounter. It’s important for us to make sure any service we offer is 100% compatible with industry standards, and K3s allows us to do just that but with simplicity and speed for our users.

Shipa is excited to launch our new webcast series, Talking Shipa. To kick this series off, we sat down with Shipa Founder and CEO, Bruno Andrade, to discuss the release of Shipa Application Management Framework for Kubernetes, version 1.2. In this video, Bruno spends a few minutes with us to talk about the new features and improvements that are packed into this new release.

Organizations have now seen the value of building microservices. They are delivering applications as discrete functional parts, each of which can be delivered as a container or service and managed separately. But for every application, there are more parts to manage than ever before, especially at scale, and that’s where many turn to an orchestrator for help.

Portainer is an open source tool that allows for container deployment and management without the need to write code. In their recent publication, ‘How to deploy Portainer on MicroK8s’, the Portainer team share with the community how easy and fast it is to deploy Portainer on MicroK8s. In fact, the entire process only requires a single command! For a step-by-step walkthrough of the process, take a look at Portainer’s 5 minute video below. Install MicroK8s

I have seen many junior data scientists and machine learning engineers start a new job or a consulting engagement for a telecom company coming from different industries and thinking that it’s yet another project like many others. What they usually don’t know is that “It’s a trap!”. I spent several years forging telecom data into valuable insights, and looking back, there are a couple of things I would have loved to know at the beginning of my journey.

Migrating to the cloud is a significant, complicated endeavor, one that requires a realistic migration plan for any application portfolios that will be mapped out first. To get started, a detailed technical analysis of each application's cloud readiness helps determine the best cloud migration approach and strategy to take. If this sounds like a daunting process, that’s because it often is! Let's understand why.

The defense evasion category inside MITRE ATT&CK covers several techniques an attacker can use to avoid getting caught. Familiarizing yourself with these techniques will help secure your infrastructure. MITRE ATT&CK is a comprehensive knowledge base that analyzes all of the tactics, techniques, and procedures (TTPs) that advanced threat actors could possibly use in their attacks. Rather than a compliance standard, it is a framework that serves as a foundation for threat models and methodologies.

Data streaming is important for getting insights in real time and reacting to events as fast as possible. Its application is wide, from banking transactions and website click analytics to IoT devices and motorsports. The last example represents a really interesting use case.

Kubernetes provides abstraction and simplicity with a declarative model to program complex deployments. However, this abstraction and simplicity create complexity when debugging microservices in this abstract layer. The following four vectors make it challenging to troubleshoot microservices.

In this episode of Coffee & Containers, Bruno Andrade, Founder/CEO @ Shipa and Melissa McKay, Developer Advocate @ JFrog discuss some specific Kubernetes related obstacles.