Operations | Monitoring | ITSM | DevOps | Cloud

March 2022

Why 87% of AI/ML Projects Never Make It Into Production-And How to Fix It

Going from prototype to production is perilous when it comes to artificial intelligence (AI) and machine learning (ML). However, many organizations struggle moving from a prototype on a single machine to a scalable, production-grade deployment. In fact, research has found that the vast majority—87%—of AI projects never make it into production. And for the few models that are ever deployed, it takes 90 days or more to get there.

Networking, security & observability with Cilium

Raymond de Jong, Senior Solutions Architect at Isovalent, will be our guest as we explore Cilium - a BPF powered open-source Cloud Native Networking solution, providing security, observability, scalability, and superior performance. Civo's own Kunal Kushwaha will also look at using Cilium for network policy security on Civo Kubernetes.

Your First Dagger Kubernetes Deployment with Shipa

The DevOps and Platform Engineering space certainly is one that evolves fast. As new development paradigms get consumed, supporting the development pipeline is crucial. Pushing a public release of v0.2.x on March 30th, 2022, Dagger, from the creators of Docker, is another approach in portability and consistency in CI/CD pipelines. What the Docker Container has done applications, Dagger is hoping to achieve that with CI/CD pipelines.

Managing Sensitive Data in Kubernetes with Sealed Secrets and External Secrets Operator (ESO)

Having multiple environments that can be dynamically configured has become akin to modern software development. This is especially true in an enterprise context where the software release cycles typically consist of separate compute environments like dev, stage and production. These environments are usually distinguished by data that drives the specific behavior of the application.

A visual guide to Calico eBPF data plane validation

In previous blog posts, my colleagues and I have introduced and explored the Calico eBPF data plane in detail, including learning how to validate that it is configured and running correctly. If you have the time, those are still a great read; you could dive in with the Calico eBPF Data Plane Deep-Dive.

How to automate verification of deployments with Argo Rollouts and Elastic Observability

Shipping complex applications at high velocity lead to increased failures. Longer pipelines, scattered microservices, and more code inherently lead to bigger complexity where small mistakes may cost you big time.

Sentry Points of Presence: How We Built a Distributed Ingestion Infrastructure

Event ingestion is one of the most mission-critical components at Sentry, so it’s only natural that we constantly strive to improve its scalability and efficiency. In this blog post, we want to share our journey of designing and building a distributed ingestion infrastructure—Sentry Points of Presence— that handles billions of events per day and helps thousands of organizations see what actually matters and solve critical issues quickly.

Difference between Continuous Integration, Continuous Deployment and Continuous Delivery

Continuous integration is a DevOps practice, where developers continuously integrate the code changes into a central repository. It most often refers to the build or the integration stage of the software release process.A continuous integration service helps to automatically build and run unit tests on the new code changes to find any errors instantly.

Overcoming Kubernetes Infrastructure Challenges at the Edge of the Network

In response to the explosive growth of Internet of Things (IoT) devices, organizations are embracing edge computing systems to better access and understand the enormous amount of data produced by these devices. As the name suggests, edge computing moves some storage and computing resources out of the central data center and closer to where the data is generated at the edge of the network, whether that’s a factory floor, retail store, or automated car.

Learn the Basics of Kubernetes Persistence Management Functionality

In this webinar, Oleg Chunikhin, CTO at Kublr, walks you through the basics of Kubernetes (K8s) persistence management functionality and how it can be used to simplify managing persistent applications across different environments - in the cloud or on-premise. Oleg will use a demo environment with clusters in different clouds to show K8s persistence in action. Learn about.

AWS Migration Checklist For Startups

Suppose you are going to adopt AWS as your cloud provider. Whether you are migrating from some other cloud providers or it is your first time setting up your application’s infrastructure on the cloud, This article will be immensely beneficial for you. AWS is an industry leader in cloud innovation technologies and carries the largest market share among cloud providers.

Getting started with Juju and Charmed Operators: three awesome videos

Getting started with software can be confusing – depending on the complexity of the software, of course. Despite the extensive documentation available for Charmed Operator SDK and Juju some just prefer to watch video material to start with. So, let’s take the opportunity to have a look at available tutorials and presentations available on the Internet.

Infrastructure As Apps: The GitOps Future of Infra-as-code

Infrastructure-as-apps builds on infrastructure-as-code to a logical endpoint by bringing in principles of GitOps management. The term is something I coined in 2021 to describe an existing movement to bring infrastructure into the same lifecycle control as applications under GitOps. Examples of Infra-as-apps tools include Argo CD, Crossplane, Cluster API, Cello, or even SchemaHero for databases and the list is always growing. Some of the benefits of infra-as-apps include Read on to understand why.

VMware Tanzu Community Edition Taps in Cartographer for Building Secure Adaptable Cloud Native Supply Chains

The latest update to the VMware Tanzu Community Edition further streamlines the path to production with the addition of Cartographer, an open source project to build and manage modern secure software supply chains.

How to Run Java Inside Docker: Best Practices for Building Containerized Web Applications [Tutorial]

Containers are no longer a thing of the future – they are all around us. Companies use them to run everything – from the simplest scripts to large applications. You create a container and run the same thing locally, in the test environment, in QA, and finally in production. A stateless box built with minimal requirements and unlike virtual machines – without the need of virtualizing the whole operating system.

Interacting With Your First Shipa API Call with Postman

The beauty of Shipa is that no matter how the surrounding ecosystem changes e.g your Continuous Delivery or Infrastructure-as-Code stacks, the Shipa API stays the same. If you are curious about interacting with this mystical API, there are a lot of surrounding integrations that do that for you. Though, if you want to directly interact with the API, you can send out HTTP requests to the Shipa API itself to create any sort of integration you require.

The Top 6 Challenges Enterprises Face Deploying Kubernetes in Hybrid Cloud Environments

In a relatively short amount of time, Kubernetes has evolved from an internal container orchestration tool at Google to the most important cloud-native technology across the world. Its rise in popularity has made Kubernetes the preferred way to build new software experiences and modernize existing applications at scale in the cloud.

ValidKube Update: Adding Polaris to Auto-Audit K8s YAMLs

A month and a half ago we released ValidKube, our first OS project that fused the capabilities of three other popular OS tools (kubeval, kubectl-neat and trivy) in a single easy-to-use microsite. Using the microsite, any user could ensure the security and hygiene of their K8s YAML, with just a few clicks of the button, pretty much on the fly. ValidKube was born out of a straightforward concept and we were happy to see its user-friendly approach resonate almost immediately.

Firmus Supercloud sets a new standard for sustainable cloud computing with Canonical's open infrastructure

March 29th, 2022—Canonical, the publisher of Ubuntu, announces that Firmus, the Australian cloud infrastructure provider that is revolutionising data centre technology, has built its ultra-efficient and sustainable public cloud on Canonical’s Charmed OpenStack and Charmed Kubernetes.

How Girls Who Code Accelerated Kubernetes Adoption During the COVID-19 Pandemic

Running Kubernetes in production at scale can be a huge challenge for today’s organizations. And few companies have the right platform, experience, and skills to get there themselves. This was the case with Girls Who Code, an international nonprofit organization working to close the gender gap in technology, who had to quickly change course and develop things that weren’t on their radar months ago because of the COVID-19 pandemic.

Kubernetes Cloud Deployments with Terraform

Kubernetes is a rich ecosystem, and the native YAML or JSON manifest files remain a popular way to deploy applications. YAML’s support for multi-document files makes it often possible to describe complex applications with a single file. The Kubernetes CLI also allows for many individual YAML or JSON files to be applied at once by referencing their parent directory, reducing most Kubernetes deployments to a single kubectl call.

Best Practices For Deploying Web Applications

Deployment is an essential stage of any software development project. With the fast-paced modern agile software development needs, deployment frequency increases rapidly. Although it looks very appealing to often release due to its positive impact on customer satisfaction and improved customer engagement, it is risky at the same time. What if the deployment goes wrong on production?

Kubernetes Easy Button - Running Your JS Apps on Kubernetes with Shipa

Kubernetes is becoming a dominant platform for running workloads. As the Kubernetes ecosystem continues to advance capturing a wider swath of workloads, eventually your code might be headed to Kubernetes. As a Tech Lead at Shipa responsible for front-end engineering e.g what you see on the screen, my job crosses JavaScript Frameworks and Kubernetes on a daily basis.

Simplify, Secure, and Optimize your Multi-cloud Container Infrastructure with VMware Tanzu for Kubernetes Operations

Ning Ge and Keith Miracle co-wrote this post. Amidst many social and economic disruptions that have arisen in the last few years, enterprises have been forced to quicken the pace of their digital transformation initiatives, adding and consuming cloud-based capacity and capability just to stay competitive, relevant, and, for some, in business.

Kubernetes Master Class: Creating RKE2 Cluster Templates

Rancher 2.6 introduces a new Cluster-API based provisioning mechanism for RKE2 and K3s clusters. This also brings a completely new cluster templating system, which is based on Helm charts and is much more flexible compared to the old RKE1 cluster templates. In this master class, you will learn how the Cluster API works, how you can leverage it in Helm Charts, how to do versioning and how to create a nice UI wizard for them.

What Are the Differences Between Elastic Beanstalk, EKS, ECS, EC2, Lambda, and Fargate?

Life before containerization was a sore spot for developers. The satisfaction of writing code was constantly overshadowed by the frustration of attempting to force code into production. For many, deployments meant hours of reconfiguring libraries and dependencies for each environment. It was a tedious process prone to error, and it led to a lot of rework. Today, developers can deploy code using new technology such as cloud computing, containers, and container orchestration.

Rancher Desktop Now Includes The Rancher Dashboard

With the 1.2.0 release of Rancher Desktop, there are two new features available as a Feature Preview. Rancher, the multi-cluster Kubernetes manager, includes a dashboard which enables you see and interact with resources in a Kubernetes cluster. Rancher Desktop now includes this dashboard. The dashboard will enable you to view and interact with resources in your local cluster provided by Rancher Desktop.

Don't Forget About Kubernetes Jobs - Shipa Jobs Support

When I was making my first switch from a product engineering team to being field facing software engineer, one of my first projects was an integration project for a federal agency. The very first piece of enterprise software minus my productivity and development suite I was exposed to was BMC’s Control-M about 15 years ago. A lot of batch files to extract and transform data had to be run in order and on a daily basis; Control-M at the time was a job runner.

How to Model Your Gitops Environments and Promote Releases between Them

Two of the most important questions that people ask themselves on day 2 after adopting GitOps are: In the previous article of the series, I focused on what NOT to do and explained why using Git branches for different environments is a bad idea. I also hinted that the “environment-per-folder” approach is a better idea. This article has proved hugely popular and several people wanted to see all the details about the suggested structure for environments when folders are used.

How to Scale your AWS Infrastructure - Part 2

Welcome to the second post in a series of “How to Scale your AWS Infrastructure”. In the first post, we talked about horizontal scaling, autoscaling, CI/CD, infrastructure automation, containerization, etc. In this post, we will continue the discussion around databases, loose coupling, caching, CDN, etc. Let’s start the discussion with database scaling.

What's new in Sysdig - March 2022

Welcome to another iteration of What’s New in Sysdig in 2022! The “What’s new in Sysdig” blog has fallen to me, Jason Donahue, for the month of March! I am a Solutions Engineer based in New Jersey and a member of the Sysdig US East Enterprise team since September, 2021. I have worn many hats in my career, from Networking to Systems Administration to Software Engineer.

Join the Smart Cloud-Native Revolution

We are in the midst of a digital revolution that started with the PC, Internet, and mobile phone and has continued to accelerate. In this current wave, the cloud, Kubernetes, artificial intelligence (AI), and intelligent automation are combining to create the next major disruption, which we call smart cloud-native. Smart cloud-native is a powerful force that is transforming data centers, workforces, customer experiences, and the way enterprises do business.

How to Kustomize your Codefresh/Argo Runtime

The Codefresh Software Delivery Platform (CSDP) brings together the complete open source Argo toolset (Workflows, Events, CD, and Rollouts) into a single platform for enhanced efficiency and visibility of software deployments at massive scale. If you’re a new CSDP user, one of the first things you’ll do is install the CSDP runtime in one of your Kubernetes clusters.

6 Metrics to Watch for on Your Kubernetes Cluster

Kubernetes. Nowadays it seems companies in the industry are divided into two pools: those that already use it heavily for their production workloads and those that are migrating their workloads into it. The issue with Kubernetes is that it is not a single system the way Redis RabbitMQ or PostgreSQL are. It is a combination of several control plane components (for example etcd, api server) that run our workloads on the user (data) plane over a fleet of VMs.

Centrally Manage, Secure, and Monitor Kubernetes using VMware Tanzu for Kubernetes Operations

Kubernetes has become the de facto platform for running containerized workloads. Kubernetes brings a set of APIs for managing applications that can work with multiple infrastructure/cloud providers. Whether you want to deploy a containerized application on vSphere, AWS, or Azure, as long as Kubernetes is deployed in these environments, the API being used to request a container deployment stays the same. This helps application development teams tremendously.

5 takeaways from the CNCF Annual Survey 2021

The CNCF Annual Survey 2021 is in and makes for some very encouraging reading for the future of Kubernetes and its place in the tech landscape. The 2021 survey was the biggest yet, with some 3,829 developers, engineers, architects, and C-level execs in the cloud native space taking part. Here are some of our key takeaways…

How to Scale your AWS Infrastructure - Part 1

When designing a solution, you should keep future needs in mind. If the number of users increases dramatically in a short period of time, the solution should be scalable enough to handle the new growth. Making systems scalable on cloud is relatively easier as compared to scaling on-premises infrastructure. AWS has provided excellent tools/services to enable your applications for as much scalability as you want.

Quick-Start Guide to Using VMware Tanzu Mission Control and vSphere with Tanzu Services

Explosive growth of web traffic and services is forcing organizations to modernize and optimize their infrastructures. Kubernetes is core to the strategy and modernization story, but it’s only one piece. As VMware engages with its customers, significant complexities and resource needs arise that are not always apparent in the planning stages of Kubernetes deployments. The complexity of even a single deployment can introduce delays and slow projects to a crawl.

Should Your Startup Use AWS Managed Services?

Let’s face it. Gaining a competitive advantage in the target market is expensive. Even if you have a good idea and its execution plan in mind, operations related to management, storage, networking, service provisioning, security, and application management will cost you a fortune. To say the least, a cutting-edge IT infrastructure, a reliable team, and a strategy for rapid product releases or expansion/scaling is a must for your product’s success.

Pulumi or Terraform for applications? Maybe, both?

Cloud-native is an evolving architecture. Existing vendors will keep on evolving their offerings and different teams inside your organization should be able to use the tool that will support them better in delivering their desired outcome fast. By implementing a standard application layer, you enable teams to adopt what works best for them while the DevOps team can focus on adopting the infrastructure components they believe to be best to support their organization.

Ocean for Apache Spark goes GA on AWS

When Apache Spark introduced native support for Kubernetes it was a game changer for big data. Speed, scale and flexibility are now at the fingertips of data teams—-if they can master Kubernetes. It’s an uphill climb for even experienced DevOps teams. At Spot by NetApp, we’ve seen first-hand the challenges that companies are facing as they navigate the complexities of operating large-scale Kubernetes applications.

What are Linux containers?

Over the last decade, containers have become an essential part of running infrastructure more efficiently. Containers enable productivity, automation, and cost-effective deployments. But there are different types of containers to consider, and this blog explains what Linux containers are, and how they differ from application containers.

Contributor's Box (Level 1) - Unboxing the Codefresh Open Source Maintainer's

As we work diligently on transforming Codefresh into an Open Source company, we created THE MAINTAINER'S CLUB. The Maintainer's Club is a set of incentives and onramps to becoming more active in the open source community, specifically the Argo Project. There are three levels 1) Contributor 2) Member 3) Maintainer In this video, Dan Garfield, Co-Founder and Chief Open Source Officer unboxes the level 1 or Contributor Box. Check it out!

Managed and Unmanaged Clusters in VMware Tanzu Community Edition: What You Need to Know

With VMware Tanzu Community Edition you can create managed and unmanaged Kubernetes clusters. What’s the difference? Why might you be better served by one or the other? What are typical use cases for each? In this engaging chalk talk–style video, Steve Pousty and Whitney Lee answer these questions and more.

What is a YAML? - A Box of DevOps?

I recently returned from a birthday trip to Napa Valley and got to spend some time with the Shipa Team in Palo Alto during the trip. Grabbing a coffee on my trek back to San Francisco, I overheard someone talking about YAML at the coffee shop and I had to hold back my laugh. You usually do not hear folks talking about YAML out in the public but this is San Francisco. For many engineers, YAML is a way of life.

Gaps in Kubernetes Adoption Data

The Cloud Native Computing Foundation (CNCF) recently released its annual survey on the state of Kubernetes and containers. The report highlighted the tremendous and continued growth in Kubernetes adoption, as well as some challenges that still persist. Both of these takeaways mirrored the corresponding data points from our 2021 Kubernetes in the Enterprise: Annual Report. However, as we dug into the data, we found gaps, or contradictions, between the two reports.

Introducing Epinio 0.6: Smaller, Faster *and* More Capable!

With our latest releases of Epinio, we’ve focused on making both the setup and developer experience much more streamlined. We’ve looked at where users are having issues and removed many of the roadblocks. This reduced footprint also allows for more customizability and easier long-term maintenance. If you are not familiar with Epinio, it is an application development engine for Kubernetes that lets you go from code to URL in a single step.

What is CICD Pipeline? Explanation of CICD Pipeline along with Examples.

Continuous Integration(CI) is a software development practice where developers frequently merge the code and the changes in a central repository. The important goals of continuous integration is to find and resolve the bugs more quicker, improve the software quality, and reduce the time taken to validate and release new software updates. Continuous Delivery(CD), which is done on the top of Continuous Integration and includes the practice of automating the entire software release process and builds.

Platform Engineering teams are the developer's cloud provider

Organizations rely more than ever on their engineering teams to get in front of their customers. Quickly delivering the latest functionalities to end-users in a reliable way can make or break a company these days. This need raises the pressure on engineering to deliver a scalable platform, rollout application updates faster, and manage applications efficiently once in production.

Running Serverless Applications on Kubernetes with Knative

Kubernetes provides a set of primitives to run resilient, distributed applications. It takes care of scaling and automatic failover for your application and it provides deployment patterns and APIs that allow you to automate resource management and provision new workloads.

Deploying Docker Containers on AWS: Elastic Beanstalk vs ECS vs EKS

Containerization packages a software component and its environment, dependencies, and configuration into an isolated unit called a container. That makes it possible to deploy an application consistently across different computing environments, whether on-premises or on the cloud. The concept of containerization is more than a decade old.

Civo Update - March 2022

In February we had our first online meetup of the year, 'Connecting and securing your microservices by using EnRoute.' Check it out on our YouTube channel if you missed it. Meanwhile, for Civo Shorts, David Flanagan of Pulumi explains why Civo is his service provider of choice for testing environments. Plus guides and tutorials on all things Cloud Native and Civo. Read on.

Top 12 Kubernetes Risks

What’s putting your K8s workloads at risk? You probably didn’t immediately think of memory and CPU resources—yet, these pose significant threats to cost and performance in your public cloud Kubernetes and OpenShift deployments. Learn about the top 12 K8s risks and how you can visualize the spread of risk in your containers deployment. You'll also hear a methodology for drilling down to individual misconfigurations and resolving them.

Introducing Codefresh Software Delivery Platform

Enterprises need a solution that can keep pace with innovation. The Codefresh Software Delivery Platform brings together Argo Workflows, Events, CD, and Rollouts into a unified enterprise-grade solution that equips developers for continuous delivery with confidence while leveraging GitOps best practices.

Implementing a Kubernetes Application Platform - BambooHR and Shipa

In this webinar, we talk with platform engineering leaders at BambooHR, a SaaS leader in the Human Resources space, about furthering their journey into Kubernetes. We are joined by Kelsey Hightower to help moderate and provide commentary on what he has seen in the space. As BambooHR kicks off their journey with Shipa, learn from the prospectus of the team moving the needle in engineering efficiency and developer experience.

Kubernetes vs. Docker

Container technology is changing the way we think about building, shipping, and running applications. Containers are lightweight packages of software that include everything it needs to run an application. This includes operating system components as well as libraries and other dependencies. Emerging technologies such as Docker and Kubernetes empowers organizations to deliver quality software with speed and ease.
Sponsored Post

The Best Kubernetes Monitoring Tools

In this article, you'll learn about the best Kubernetes performance monitoring tools that are currently on the market. Although there are a number of application performance monitoring solutions out there, this article covers the best options in terms of their key features, functionalities, ease of setup, and the support garnered from each of their respective communities.

Build and Deploy an Application with VMware Tanzu Community Edition

Follow along as Cora Iberkleid shows how to use open source tooling provided by VMware Tanzu Community Edition to move through the core steps of cloud native service delivery. You’ll see her leverage kpack to turn source code into container images; the Harbor registry to sign images, scan them for vulnerabilities, and provide secure registry services; and Knative Serving to simplify the deployment and running of services in a Kubernetes environment.

Step-by-Step Setup of VMware Tanzu Community Edition on AWS

In this video, Steve Pousty walks through the essential steps required to install VMware Tanzu Community Edition on AWS. Follow along as he provides useful tips and detailed explanations while he prepares his environment, then sets up both management and workload clusters on AWS from a Windows computer using WSL2. His clear presentation will help you get your own environment up and running quickly.

Kubernetes vs Nomad: What to Choose in 2022?

Kubernetes has become an enormously popular choice for containerized applications since its 2014 launch. Many software developers rely on the tool, which is now in v1.23.1. However, there are other choices on the market for container orchestration. One such tool is Nomad, originally launched in 2015. Generally pitched as an alternative to Kubernetes, Nomad, which most recently released v1.2, promotes itself as a simple, flexible option for software teams.

How to maximize K3s resource efficiency using Calico's eBPF data plane

Amazon’s custom-built Graviton processor allows users to create ARM instances in the AWS public cloud, and Rancher K3s is an excellent way to run Kubernetes in these instances. By allowing a lightweight implementation of Kubernetes optimized for ARM with a single binary, K3s simplifies the cluster initialization process down to executing a simple command.

Kubernetes Health Check Using Probes

Kubernetes is an open source container orchestration platform that significantly simplifies an application's creation and management. Distributed systems like Kubernetes can be hard to manage, as they involve many moving parts and all of them must work for the system to function. Even if a small part breaks, it needs to be detected, routed and fixed. These actions also need to be automated. Kubernetes allows us to do that with the help of readiness and liveness probes.

Shipa Cloud Operations and Practices

Shipa Cloud is how we run the Shipa control plane on behalf of users in order to give them the fastest path possible to implementing Application as Code within their clusters. You can try out Shipa Cloud for free today by going to shipa.io. Besides being the fastest way possible to get started with Shipa, it also takes away the responsibility of upgrades, maintenance, and uptime of the control plane for our users, but that responsibility doesn’t just disappear.

One Minute to Deployed on Kubernetes with Shipa

In this Shipa Shorts video, we deploy to Kubernetes in under a minute. All we had to produce is an image and Shipa takes care of the rest. No need to wire Networking Policies, Service Meshes, etc. With Shipa, you can deploy to Kubernetes without having to understand Kubernetes internals. Outside the UI, no matter your flavor of CI/CD, Shipa supports that.

Enabling simple, cost-effective Kubernetes on IBM Z with MicroK8s

Containerisation has transformed the enterprise IT landscape, driving faster, more secure, and more predictable software delivery than ever before. Thanks to technologies like Docker, building containerised applications is easy, and many businesses are working with hundreds or even thousands of containers. To effectively deploy and manage all of these microservices, a container orchestration tool is essential, and Kubernetes is the leading solution.