Immutable Infrastructure Basics

One of the biggest shifts in technology that can be attributed to cloud is the move from away from hand crafting systems to defining systems and letting automation create them. This is often referred to as Infrastructure as Code, and is manifested along a spectrum. Toward the more “cloudy” end of the spectrum is the concept of immutable infrastructure, which means that once created, infrastructure exists in the same state until it is replaced by something new, but it is never changed.

Some Examples

A good example of immutability is a Java jar file. Jar files are produced by a compilation process and is executed in a Java runtime environment. If a change is desired, the jar file isn’t changed (or more accurately the files in the jar are not modified or updated). Instead, the jar file is replaced by one with the desired changes already made.

Another example is a CD/DVD ROM (ROM (Read Only Memory) is a nod to the immutability of the medium). If an OS is written to a disc for distribution, every time that disc is used to install an OS, the resulting file system is initially identical. If an updated version of the OS is available, a new disc would be required, since what is on the disc cannot change.

What about containers

A more trendy example today is a container image. Once an image is created, the image can be used to create any number of containers, and every container created with that image is identical at the moment when it starts. A container image is a lot like the CD/DVD ROM mentioned above; it can be viewed as a file system intended to never change and which includes everything required to run the single process it was designed for.

Characteristics of Immutable Infrastructure

• The most common artifact is an image that represents a file system
• Once created, an image is never changed, but it can be replaced
• Automated means are available to produce an instance from an image (no direct human involvement required)
• Configuration is external to the image (more on that in a minute)
• Images can be tagged and maintained in a long lived repository
• An image serves only one purpose or function

What’s so bad about Mutable Infrastructure?

You might be thinking, “CD/DVD ROMs are old technology. Why would we want to go back to that?” Some of the biggest problems with mutable infrastructure include:

• Changes to long running systems are often irreversible
• Hand crafted systems are hard to reproduce
  • This means they don’t scale well
  • It also means they are kept around even when utilization is low
    • Very old systems are kept past their useful life out of fear that it could not be recreated and might be needed
• Scaling is slow and can be often problematic
• Systems diverge from their initial state resulting in inconsistent state among hosts that serve the same function
• Releases, patching and other maintenance activities are high touch and disruptive to system availability

Alternatives to Immutable Infrastructure

Some would argue that technologies like Ansible, Chef, etc. address the traditional problems mutable infrastructure. I agree that introducing a system configuration tool can mitigate some of the issues above. In practice, the amount of effort to address every aspect of a system or host life cycle is rarely invested. The end result is a system that is partially managed. I have observed that there are still frequent manual modifications to running systems, which results in divergence and a loss of consistency.

Even some technologies that start as immutable can degenerate into mutable if not managed properly. For example, I have observed some systems that were deployed from an machine image that was created as part of a pipeline, but at some point someone made a manual change to the system rather than pushing the change through the pipeline to create a new image. As a result, the system now diverged and needs to be maintained separately until the pipeline is updated to reflect the manual changes.

Keep Configuration Separate

In a future article I’ll introduce the idea of promoting images rather than deploying directly to existing hosts. The idea is that the Software Design Life Cycle (SDLC) includes a natural progression from Development to Test to Production (with a few more stops in between). During this progression, configuration often changes. A common example is database connection details, but it may include API endpoints, number of instances, thresholds, auth+auth, etc.

If any of the above configuration details are “baked in” to the image, that means a new image must be created for each environment, which breaks the immutability characteristic that the image should never change. A useful analogy is a compiled binary. If you test a specific binary image and decide it’s ready for production, but then have to change the code and compile a new binary for production, you can’t be certain that the new binary will behave exactly as the old one that was tested. It would also be inconvenient to have to compile a new binary with each configuration change, such as an updated database password.

Rule of thumb: Keep configuration outside your images so that the same image can be used in multiple contexts without modification.

Navigation

• Immutable Infrastructure Basics (you are here)
• Immutable Infrastructure: Development Environments and Team Collaboration
• Immutable Infrastructure: CI/CD (test/stage/audit/etc.)
• Immutable Infrastructure: Production Release