Daniel Watrous on Software Engineering

A Collection of Software Problems and Solutions

Posts tagged cloud native

Software Engineering

HEAT or Ansible in OpenStack? Both!

Someone asked me today whether he should use HEAT or Ansible to automate his OpenStack deployment. My answer is that he should use both! It’s helpful to understand the original design decisions for each tool in order to use each effectively. OpenStack HEAT and Ansible were designed to do different things, although in the opensource tradition, they have been extended to accommodate some overlapping functionalities.

Cloud Native

In my post on What is Cloud Native, I show the five elements of application life cycle that can be automated in the cloud (image shown below). The two life cycle elements in blue, provision and configure, correspond to the most effective use of HEAT and Ansible.


OpenStack HEAT for Provisioning

HEAT is designed to capture details related to infrastructure and accommodate provisioning of that infrastructure on OpenStack. CloudFormation does the same thing in AWS and Terraform is an abstraction that has providers for both OpenStack and AWS (and many others).

HEAT provides vocabulary to define compute, storage, network and other infrastructure related resources. This includes the interrelationships between infrastructure resources, such as associating floating IPs with compute resources or binding a compute resource to a specific network. This also includes some bookkeeping items, like assigning key pairs for authentication and naming resources.

The end result of executing a heat template is a collection of one or more infrastructure resources based on existing images (VM, or volume).

Ansible for Configuration

Ansible, on the other hand, is designed to configure infrastructure after it has been provisioned. This includes activities like installing libraries and setting up a specific run time environment. System details like firewalls and log management, as well as application stack, databases, etc. are easily managed from Ansible.

Ansible can also easily accommodate application deployment. Activities such as moving application artifacts into specific places, managing users/groups and file permissions, tweaking configuration files, etc. are all easily done in Ansible.

The end result of executing an Ansible playbook is ready-to-use infrastructure.

Where is the Overlap?

Ansible can provision resources in openstack. HEAT can send a cloud-init script to a new server to perform configuration of the server. In the case of Ansible for provisioning, it is not nearly as articulate or granular for the purpose of defining infrastructure as HEAT. In the case of HEAT configuring infrastructure through cloud-init, you still need to find some way to dynamically manage the cloud-init scripts to configure each compute resource to fit into your larger system. I do use cloud-init with HEAT, but I generally find more value in leaving the bulk of configuration to Ansible.

Ansible inventory from HEAT

When using HEAT and Ansible together, it is necessary to generate the ansible inventory file from HEAT output. To accomplish this, you want to make sure HEAT outputs necessary information, like IP addresses. You can use your favorite scripting language to query HEAT and write the inventory file.

Example using both HEAT and Ansible

A while ago I published two articles that showed how I develop the Ansible configuration, and then extend that to work with HEAT for deploying complex, multi-server environments.

Install and configure a Multi-node Hadoop cluster using Ansible

Bulid a multi-server Hadoop cluster in OpenStack in minutes

The first article lays the foundation for deploying a complex system with Ansible. The second article builds on this by introducing HEAT to provision the infrastructure. The Ansible inventory file is dynamically generated using a python script and the OpenStack CLI.


While there is some ambiguity around the term provision in cloud parlance, I consider provision to be the process of creating infrastructure resources that are not generally configured. I refer to configuration as the process of operating against those provisioned resources to prepare them for a specific use case, such as running an application or a database. HEAT is a powerful tool for provisioning resources in OpenStack and Ansible is a great fit for configuring existing infrastructure resources.

Software Engineering

What is Cloud Native?

I hear a lot of people talking about cloud native applications these days. This includes technologists and business managers. I have found that there really is a spectrum of meaning for the term cloud native and that two people rarely mean the same thing when they say cloud native.

At one end of the spectrum would be running a traditional workload on a virtual machine. In this scenario the virtual host may have been manually provisioned, manually configured, manually deployed, etc. It’s cloudiness comes from the fact that it’s a virtual machine running in the cloud.

I tend to think of cloud native at the other end and propose the following definition:

The ability to provision and configure infrastructure, stage and deploy an application and address the scale and health needs of the application in an automated and deterministic way without human interaction

The activities necessary to accomplish the above are:

  • Provision
  • Configure
  • Build and Test
  • Deploy
  • Scale and Heal


Provision and Configure

The following diagram illustrates some of the workflow involved in provisioning and configuring resources for a cloud native application.

You’ll notice that there are some abstractions listed, including HEAT for openstack, CloudFormation for AWS and even Terraform, which can provision against both openstack and AWS. You’ll also notice that I include a provision flow that produces an image rather than an actual running resource. This can be helpful when using IaaS directly, but becomes essential when using containers. The management of that image creation process should include a CI/CD pipeline and a versioned image registry (more about that another time).

Build, Test, Deploy

With provisioning defined it’s time to look at the application Build, Test and Deploy steps. These are depicted in the following figure:

The color of the “Prepare Infrastructure” activity should hint that in this process it represents the workflow shown above under Provision and Configure. For clarity, various steps have been grouped under the heading “Application Staging Process”. While these can occur independently (and unfortunately sometimes testing never happens), it’s helpful to think of those four steps as necessary to validate any potential release. It should be possible to fully automate the staging of an application.


The discovery step is often still done in a manual way using configuration files or even manual edits after deploy. Discovery could include making sure application components know how to reach a database or how a load balancer knows to which application servers it should direct traffic. In a cloud native application, this discovery should be fully automated. When using containers it will be essential and very fluid. Some mechanisms that accommodate discovery include system level tools like etcd and DNS based tools like consul.

Monitor and Heal or Scale

There are loads of monitoring tools available today. A cloud native application requires monitoring to be close to real time and needs to be able to act on monitoring outputs. This may involve creating new resources, destroying unhealthy resources and even shifting workloads around based on latency or other metrics.

Tools and Patterns

There are many tools to establish the workflows shown above. The provision step will almost always be provider specific and based on their API. Some tools, such as terraform, attempt to abstract this away from the provider with mixed results. The configure step might include Ansible or a similar tool. The build, test and deploy process will likely use a tool like Jenkins to accomplish automation. In some cases the above process may include multiple providers, all integrated by your application.

Regardless of the tools you choose, the most important characteristic of a cloud native application is that all of the activities listed are automated and deterministic.