Immutable Infrastructure: CI/CD (test/stage/audit/etc.)

This article is part of a series on Immutable Infrastructure and discusses the scenario of CI/CD pipelines to deliver high quality software quickly. It may be valuable to review Immutable Infrastructure Basics before reading this article.

Continuous Integration and Continuous Delivery, CI/CD, is a practice in software development that aims to ensure high quality updates efficiently progress toward a release. The “Integration” part of CI/CD seeks to confirm that a software change will work with related systems. The divergence of systems that occurs without immutable infrastructure can reduce the effectiveness of CI/CD pipelines. In the illustration below, two approaches are shown, Distribution for mutable infrastructure vs. Promotion for immutable infrastructure.

Promote vs. Distribute

Promote and Distribute respond to the question of how software changes are delivered (the “D” in CI/CD). This scenario explores these ideas through the lens of a single software change: software change 142.

Distribute

The most common way to deliver a software change is to distribute it to one or more long running systems. As shown in the illustration, and similar to the team laptops in the previous article, each long running host exists in a specific state. The intention is usually that these systems are homogeneous so that versions, patch levels, data, language and runtimes, etc. are identical from one host to the next. In practice, there is often divergence that occurs as systems age or are used for specific development or delivery priorities.

A discrete change, like software change 142 is usually packaged into some type of deployment artifact and copied to an existing host. Depending on what changed, it may also be necessary to update libraries, data, language runtime, etc. Changes like this are often irreversible, which means that after the target host has been modified, it can be very difficult to revert to the previous state. Each host must also be updated separately, which may result in the user experiencing downtime or a mixed state as the distribution of the change proceeds.

Promote

When software change 142 is packaged into an immutable artifact, such as a container image, delivery looks much different. New instances can be created for any purpose, such as development, test and production. These new instances do not require making any changes to existing systems, which means that any currently running instances are not disturbed.

If software change 142 is found to be a good change in development, the same image that was used in development is promoted to test. In this case, promotion refers to the image, not the instance. The image is immutable, which means that an instance tagged for test is identical to an instance tagged for development (with the exception of configuration). As the image follows the promotion path, there is never any need to modify existing instances and all new instances are identical to running instances. This virtually eliminates disruption to the end user and ensures that what is validated is exactly what makes it to production.

Routing Control

Promotion type releases follow a “replace” rather than an “update” approach. This could look like a rolling update, where existing instances are replaced one at a time by new instances. It could also look like a blue green deployment, where a full deployment of new instances is created and traffic is simply routed to the new instances when desired. The old instances can be left running or destroyed as desired.

One thing that is common among deployments of immutable artifacts is that what the end user experiences is determined by routing, not the state of any instance or host. In the traditional, mutable approach, end users are routed to a discrete list of long lived hosts. Any changes to those hosts impact the end user experience directly and immediately, even if only by reducing capacity when a host is removed from Load Balancer rotation to be updated.

A new instance created from an immutable artifact only becomes available when routing is adjusted to send traffic to it. This means there is no immediate impact to the end user. This also makes it possible to employ some advanced validation techniques, such as traffic mirroring or canary validation. What’s important to note is that routing becomes the primary control, not system access.

Navigation

• Immutable Infrastructure Basics
• Immutable Infrastructure: Development Environments and Team Collaboration
• Immutable Infrastructure: CI/CD (test/stage/audit/etc.) (you are here)
• Immutable Infrastructure: Production Release