Do Humans or Machines consume IaaS (REST API)

Infrastructure as a Service, like OpenStack and AWS, have made it possible to consume infrastructure on demand. It’s important to understand the ways in which both humans and machines interact with IaaS offerings in order to design optimal systems that leverage all possible automation opportunities. I drew the diagram below to help illustrate. Everything is […]

Read more

HEAT or Ansible in OpenStack? Both!

Someone asked me today whether he should use HEAT or Ansible to automate his OpenStack deployment. My answer is that he should use both! It’s helpful to understand the original design decisions for each tool in order to use each effectively. OpenStack HEAT and Ansible were designed to do different things, although in the opensource […]

Read more

Deploy MongoDB using Ansible

I’ve recently had some people ask how I deploy MongoDB. For a while I used their excellent online tool to deploy and monitor my clusters. Unfortunately they changed direction and I couldn’t afford their new tools, so I turned to Ansible. In order more easily share the process, I posted a simple example that you […]

Read more

Create better software by starting on paper

Today I read an article on the Wall Street Journal about the benefits of taking handwritten notes over taking notes electronically. It’s a great article and I have seen parallels in my own professional work. For years I have trained computer programmers and others in various technology roles. I have observed a similar effect which […]

Read more

What is Cloud Native?

I hear a lot of people talking about cloud native applications these days. This includes technologists and business managers. I have found that there really is a spectrum of meaning for the term cloud native and that two people rarely mean the same thing when they say cloud native. At one end of the spectrum […]

Read more

Microservices are not as new as you think

All around me architects and business managers are beginning to mandate that internal software applications be built (and sometimes rebuilt) as microservices so that we can reuse them and compose applications more quickly. I do admit that the idea of a curated catalog of well designed microservices is attractive. Contrary to all the buzz that […]

Read more

Bulid a multi-server Hadoop cluster in OpenStack in minutes

In a previous post I demonstrated a method to deploy a multi-node Hadoop cluster using Vagrant and Ansible. This post builds on that and shows how to deploy a Hadoop cluster with an arbitrary number of slave nodes in minutes on OpenStack. This process makes use of the OpenStack orchestration layer HEAT to provision the […]

Read more

Install and configure a Multi-node Hadoop cluster using Ansible

I’ve recently been involved with several groups interested in using Hadoop to process large sets of data, including use of higher level abstractions on top of Hadoop like Pig and Hive. What has surprised me most is that no one is automating their installation of Hadoop. In each case that I’ve observed they start by […]

Read more

Configuration of python web applications

Hopefully it’s obvious that separating configuration from application code is always a good idea. One simple and effective way I’ve found to do this in my python (think bottle, flask, etc.) apps is with a simple JSON configuration file. Choosing JSON makes sense for a few reasons: Easy to read (for humans) Easy to consume […]

Read more

Encryption of secrets in source code (AESCrypt + Ansible)

The more I automate, the more I have to answer the question of how to manage my secrets. Secrets that frequently come up include: SSH key pairs SSL private keys Credentials for external resources, such as databases and SaaS integrations Before cloud, when server resources were not ephemeral, these could be managed manually when the […]

Read more