Operations | Monitoring | ITSM | DevOps | Cloud

October 2021

Continuous deployment of Node apps to Heroku

CircleCI orbs are reusable packages of YAML configuration that condense repeated pieces of config into a single line of code. Since its launch in 2018, the CircleCI orbs registry has been used by developers, development teams, and by companies who want to help developers integrate their services seamlessly into continuous integration pipelines. In this tutorial, we will show how to use CircleCI orbs to continuously deploy a Node.js application to Heroku, one of the most popular hosting platforms.

A CI/CD Template for Terraform

Continuous integration (CI) makes the cycle from design to code to building artifacts seamless and consistent. Continuous delivery (CD) makes delivery of that artifact to an environment the same every time. But, what about the actual environment the artifact is running in? Is it the same every time? That’s a hard thing to guarantee — unless you take advantage of an Infrastructure-as-Code (IaC) approach. This post explains how to use Infrastructure-as-Code to improve CI/CD.

Writing Ansible Playbooks for New Terraform Servers

Over the past few years, cloud computing has enabled agile, dynamic management of software and hardware components, on-demand. Nowadays, we can define our desired infrastructure in as little as a few lines of code, and we can provision real servers on cloud providers like AWS or Azure. Terraform is an open-source infrastructure-as-code (IaC) tool that has become the de facto solution for provisioning one aspect of those components.

Config best practices: dependency caching

Let’s face it: Creating the optimal CI/CD workflow is not always a simple task. In fact, writing effective and efficient configuration code is the biggest hurdle that many developers face in their DevOps journey. But you don’t need to be an expert to set up a fast, reliable testing and deployment infrastructure. With a few straightforward techniques, you can optimize your config.yml file and unleash the full potential of your CI/CD pipelines.

Deploy Iron Bank-Approved Artifactory/Xray on AWS GovCloud and RKE2

With Artifactory and Xray now included in the U.S. Department of Defense’s Iron Bank container repository, we’re eager to help you benefit from this accreditation. Today, we’ll explain how to deploy these hardened JFrog images on AWS GovCloud using Rancher Kubernetes Edition (RKE2.) Specifically, we’ll describe the installation and configuration of the Iron Bank-accredited Artifactory version 7.21.7 and Xray version 3.30.2.

Dead Evil: A Software Supply Chain Possession

Deep in the woods, where trees are black and the air is thick, steam rises wistfully across the damp ground. A single dirt track, barely wide enough to pass, scars the terrain for what seems like an endless number of miles. It winds its way through the mountains and valleys, across a rickety bridge over a cavernous ravine, before plunging back into darkness, the trees bending over as if to grasp those passing through. Finally, in a small clearing, a lonely decrepit wooden cabin reveals itself.

Package Delivery Networks: How They Differ From CDNs

A crucial part of effective package management is package distribution. Whether you are dealing with distributed development teams, deploying a distributed application or even if you are a software vendor, you need efficient, performant and reliable delivery of your software packages or artifacts. And for that, you need infrastructure. Lots of infrastructure. To deliver software globally, at low latencies, you’ll need infrastructure in many regions, preferably as many as possible.

Introducing Test Insights with flaky test detection

The CircleCI Insights dashboard was designed to help you improve your delivery efficiency. We launched the dashboard a year ago to provide teams with actionable data for optimizing your pipelines. Since then, we’ve been listening to your feedback. By far, the most requested functionality is the ability to gain further visibility into test performance.

A guide to personal retrospectives in engineering

Retrospectives are a well-established resource in the software and systems engineering toolbox. From sprint retros through to post-incident reviews, we look back on our work to learn from it and to get better. We can apply the same ideas to our professional practice with a personal retrospective: writing an analysis of our experiences to learn as much as possible. We could look over a whole year of work, or focus more closely on a particular project.

Monitor your CircleCI environment with Datadog

Datadog CI Visibility provides a unified platform for monitoring your CI/CD pipelines. Now, we are partnering with CircleCI to extend that same critical visibility to your CircleCI environment. Datadog’s integration uses CircleCI webhooks to capture information about the status and performance of your workflows and associated jobs, such as a job’s duration and whether or not it failed or was canceled.

API contract testing with Joi

When you sign a contract, you expect both parties to hold their end of the bargain. The same can be true for testing applications. Contract testing is a way to make sure that services can communicate with each other and that the data shared between the services is consistent with a specified set of rules. In this post, I will guide you through using Joi as a library to create API contracts for services consuming an API.

Metrics for improved Docker container management and performance

When running a cloud service, it’s never good for customers to be the first people noticing an issue. It happened to our customers over the course of a few months, and we began to accumulate a series of reports of unpredictable start-up times for Docker jobs. At first the reports were rare, but the frequency began to increase. Jobs with high parallelism were disproportionately represented in the reports.

New Xray Features Enhance Workflows, Productivity and UX

The recently released JFrog Xray versions 3.31 & 3.32 have brought to the table a raft of new capabilities designed to improve and streamline your workflows, productivity and user experience. The new features, detailed below, solidify Xray as the optimum universal software composition analysis (SCA) solution for JFrog Artifactory that’s trusted by developers and DevSecOps teams to identify and eliminate open source software vulnerabilities and license compliance violations from their releases.

The What and The Why of Cloud Native Applications - An Introductory Guide

Companies across industries are under tremendous pressure to develop and deploy IT applications and services faster and with far greater efficiency. Traditional enterprise application development falls short since it is not efficient and speedy. IT and business leaders are keen to take advantage of cloud computing as it offers businesses cost savings, scalability at the touch of a button, and flexibility to respond quickly to change.

Making CI/CD Work with Serverless

As a developer, serverless lets you concentrate on what you do best: building your product. What happens when we want to implement a CI/CD flow with the serverless mindset? A supercharged CI/CD flow. In this webinar, AWS Serverless Hero and Lumigo VP Engineering Efi Merdler-Kravitz presents Lumigo’s own journey in building a 100% serverless CI/CD pipeline.

JFrog Cold Artifact Storage: Retention Policies for Your Binaries

With the trend towards smaller but more frequent software releases, your binaries and artifacts keep accumulating faster. Our enterprise customers each maintain an average of 20 million unique artifacts, adding 130% more each year. Eventually, a clutter of outdated binaries forms, and fInding the binaries you need becomes unwieldy, difficult, and confusing. Over time, your artifact repository’s performance can suffer from degradation.

Use these metrics to get the most out of your engineering team

I’ve been leading software teams for more than 20 years and one thing I’ve learned about metrics is that leaders tend to put too much emphasis on engineering metrics alone, without considering the bigger picture. After speaking to a range of engineering industry leaders, and poring over millions of jobs processed from software teams worldwide, we found that the most insightful and relevant metrics fall into three categories: What metrics are meaningful for your team to measure?

CVE-2020-27304 - RCE via Directory Traversal in CivetWeb HTTP server

JFrog has recently disclosed a directory traversal issue in CivetWeb, a very popular embeddable web server/library that can either be used as a standalone web server or included as a library to add web server functionality to an existing application. The issue has been assigned to CVE-2020-27304.

Faster CI Builds with Docker Remote Caching

Bitbucket Pipelines provides a Docker caching feature that can help improve build times. However, the limitation is that only compressed caches under 1GB are saved and can be used. In this blog, we outline a process showing how you can use compressed caches that are larger than 1GB. With Docker versions >= 19.03, you can use the BuildKit feature. With BuildKit, you don’t need to keep the cache locally before building the Docker image since it caches each build layer in your image registry.

GitLab vs JFrog: Who Has the Right Stuff?

Like the historic space race, the competition to plant the flag of DevOps is blasting off. According to market intelligence firm IDC, global business will invest $6.8 trillion in digital transformation by 2023. Yet research also suggests that 70 percent of them will fail to meet their goals. JFrog was the first company to offer a universal, hybrid, end-to-end DevOps platform.

Smoke testing in CI/CD pipelines

Here’s a common situation that plagues many development teams. You run an application through your CI/CD pipeline and all of the tests pass, which is great. But when you deploy it to a live target environment the application just does not function as expected. You can’t always predict what will happen when your application is pushed live. The solution?

Introducing default values for custom pipeline variables

Support for including default values in custom pipelines has been a highly requested feature. We are happy to announce that this feature is now live. Providing a default value helps avoid errors when you manually trigger a custom pipeline. If you often rely on the same value for certain variables, it can be frustrating to get a failed build when you forget to specify the value or have a typo when providing the value.

Dynamically rendering config templates for secrets management

It’s often necessary to inject secrets into your build or deployment process so that the deployed service can interact with other services. This can be straightforward if you’re only deploying to a single environment. When deploying to multiple environments, though, you might need to dynamically inject different secrets depending on the environment to which you’re deploying.

The Confident Commit | Ep. 13 Embrace simplicity: the #1 rule for software leaders

Rob sits down with Deepak Giridharagopal, CTO of Puppet to discuss the lessons they've learned through years as software leaders. Deepak shares the history and trajectory of Puppet, navigating shifting roles as your company scales, and his top leadership practices learned. Rob and Deepak dive into how to implement simplicity when complexity is often favored. Tune in today! And don't forget to Like and Subscribe to The Confident Commit podcast playlist for alerts to new episodes published biweekly.

What Is CI/CD and How to Build ETL Processes

In today’s world of heterogeneous data ecosystems, managing and consuming data can be cumbersome. Organizations often have multiple systems of truth in corresponding to the applications managing the data. While data engineers dream of software that would make it easy to consume and digest different data streams from disparate systems, that scenario rarely comes to fruition.

Topio + JFrog | Scaling Continuous Software Delivery for Edge & IoT Applications

Enterprises came to expect that software that underpins their business can change at the speed of the market. Cloud-native technologies and modern DevOps tools enable enterprises to continuously deploy software updates across data centers and public clouds. But things often get slowed down when trying to update applications across mixed environments and large fleets of edges and IoT devices.

Well-tested code: in search of meaningful coverage

If you work anywhere near the field of software development, you’ve likely already heard that you should always write code that is well-tested. Everyone wants to have well-tested code and for a good reason! Testing ensures our code is working as intended and protects against regression. Thoroughly testing code helps teams confidently ship software faster and with fewer issues.

Introducing Codefresh Software Delivery Platform powered by Argo

With KubeCon 2021 upon us, we look forward to seeing many exciting announcements from our peers in the open-source DevOps community in the days ahead. Codefresh is honored to make some exciting news of our own. Today we officially unveiled the Codefresh Argo Platform – a fully featured, enterprise-class implementation of Argo.

How to easily track DORA metrics

With Sleuth, you can instantly and automatically track the four DORA metrics: Change Lead Time, Deploy Frequency, Change Failure Rate, and MTTR. No more, no less. Sleuth tracks them accurately, doesn't attempt to track dubious individual metrics, and doesn't require any manual work. SLEUTH A deploy-based Accelerate Metrics tracker both managers and developers love.

How to Reduce Change Lead Time

The time it takes to get a change into production, also known as Change Lead Time, is an important measure of developer productivity, and one of the four DORA metrics. In Sleuth, you can easily break down Change Lead Time into activities to get insights into potential bottlenecks. SLEUTH A deploy-based Accelerate Metrics tracker both managers and developers love.

JFrog and Upswift: Bringing IoT Software Updates to DevOps Upswift Acquistion

JFrog has acquired Upswift to bring the world of connected devices into the DevOps pipeline! Managing fleets of devices and edge applications remotely - including over-the-air (OTA) updates, security, monitoring, controlling and more - has quickly become unwieldy for most companies, with growth of connected devices expected to reach 24 billion in 2026. But, most of today’s DevOps solutions are not optimized or built to deliver software updates to distributed edge and IoT environments.

Tales of A11y In Grafana OS: Introducing Pa11y CI into our pipeline by Alexa Vargas

We want to make Grafana accessible to everyone! In this talk, Alexa will share how Grafana recently introduced Pa11y CI into the Grafana Continuous Integration pipeline. The library supports our developers and contributors to highlight a11y issues. And more importantly, it acts as a gatekeeper, stopping new A11y issues from making it into the project. You will additionally hear about the alternatives that were considered and their challenges. This talk will have everything!

An intro to Infrastructure as Code

Infrastructure as Code (IaC) is the practice of recording the desired state of your infrastructure using a declarative language. In this article, I’m going to assume that your team is starting from scratch. Maybe some of your build process has been scripted, and maybe there is some manual testing and quality assurance work happening. Many readers will find that they are midway through the IaC adoption journey I’ll describe, or that they have missed some steps.

Proceed With Care: How to Use Approval Gates in Pipelines

While DevOps automation aims to eliminate most human intervention in the CI/CD DevOps pipeline, you can’t always cut people completely out of the process. There are still times when you’ll want an expert, hands-on review to assure that everything is as it should be before allowing your pipeline to proceed further.

Debugging CI/CD pipelines with SSH access

In my interactions at industry events like AWS re:invent and KubeCon, I talk with a lot of developers. Devs often tell stories of things that prevent them from working quickly and efficiently. Many involve frustrating interactions with sys admins, SREs, or DevOps colleagues. One story I have heard several times involves a conversation like this: dev: Hey, SRE team. My build is failing and I don’t know what’s happening with the app in the build node.

SOA vs microservices: going beyond the monolith

Modern software development increasingly relies on distributed, service-based architectural patterns to achieve scalability, reliability, and rapid build, test, and release cycles. Two of the most popular service-based approaches are service-oriented architecture (SOA) and microservices. In this article, we will examine both approaches to identify their similarities and differences as well as some use cases for each.

Building Kotlin Multiplatform projects in a CI/CD pipeline

Kotlin is one of the most versatile programming languages available, in large part because of the Kotlin team’s focus on bringing it to as many platforms as possible. It is the primary language for developing Android applications and is popular for JVM backends. Kotlin also features targets for native binary compilation with Kotlin/Native, and for web through Kotlin/JS. One of its most promising features is the ability to target multiple platforms it compiles to.

The History of CI/CD

When you’re new to an industry, you encounter a lot of new concepts. This is especially true with DevOps, a fairly young corner of tech where things move very quickly, by design. Some of the concepts we consider central to DevOps are actually pretty old, though, predating the birth of DevOps by a decade or more. Without this context for how things evolved, and for the specific ways in which software development was more difficult without the methodologies and toolsets we have today, grasping the "why" for modern abstractions can be difficult. Without understanding the "why," learning to use a new tool well isn’t as easy as it could be.

23andMe's Yamale Python code injection, and properly sanitizing eval()

JFrog security research team (formerly Vdoo) has recently disclosed a code injection issue in Yamale, a popular schema validator for YAML that’s used by over 200 repositories. The issue has been assigned to CVE-2021-38305.

Get Cybersmart with JFrog This October

We live in a world of increasingly connected devices – phones, digital assistants, smart watches, cars, thermostats, refrigerators, windmills, and more. More than 50% of the world’s population is now online and two-thirds own a mobile device, according to the World Economic Forum. Additionally, the codebase of today’s applications typically consists mainly of open source components – exposing them to greater risk of hacking than ever before.

The Vulnerability Conundrum: Improving the Disclosure Process

The vulnerability disclosure process involves reporting security flaws in software or hardware, and can be complex. Cooperation between the organization responsible for the software or hardware, and the security researcher who discovers the vulnerability can be complicated. In this blog we’ll look at the vulnerability disclosure process, the parties involved and how they can collaborate productively.

Fold Your Repos Into PHP Composer v2 with Artifactory

If you’re among the nearly one in four professional developers using PHP (according to StackOverflow’s 2021 survey), then the maintainers of Composer would really like you to migrate from v1 of the PHP package manager to v2. On October 24 2020, Composer 2.0.0 was released with some major improvements.Since almost eight out of every ten websites on the internet use PHP in some way, that’s a change with big impact.