Configuration of python web applications

July 9, 2015

Software Engineering

No Comments

Daniel Watrous

Hopefully it’s obvious that separating configuration from application code is always a good idea. One simple and effective way I’ve found to do this in my python (think bottle, flask, etc.) apps is with a simple JSON configuration file. Choosing JSON makes sense for a few reasons:

Easy to read (for humans)
Easy to consume (by your application
Can be version alongside application code
Can be turned into a configuration REST service

Here’s a short example of how to do this for a simple python application that uses MongoDB. First the configuration file.

{
    "use_ssl": false,
    "authentication": {
        "enabled": false,
        "username": "myuser",
        "password": "secret"},
    "mongodb_port": 27017,
    "mongodb_host": "localhost",
    "mongodb_dbname": "appname"
}

Now in the python application you would load the simple JSON file above in a single line:

# load in configuration
configuration = json.load(open('softwarelicense.conf.json', 'r'))

Now in your connection code, you would reference the configuration object.

# establish connection pool and a handle to the database
mongo_client = MongoClient(configuration['mongodb_host'], configuration['mongodb_port'], ssl=configuration['use_ssl'])
db = mongo_client[configuration['mongodb_dbname']]
if configuration['authentication']['enabled']:
    db.authenticate(configuration['authentication']['username'], configuration['authentication']['password'])
 
# get handle to collections in MongoDB
mycollection = db.mycollection

This keeps the magic out of your application and makes it easy to change your configuration without direct changes to your application code.

Configuration as a web service

Building a web service to return configuration data is more complicated for a few reasons. One is that you want to make sure it’s secure, since most configuration data involves credentials, secrets and details about deployment details (e.g. paths). If a hacker managed to get your configuration data that could significantly increase his attack surface. Another reason is that it’s another potential point of failure. If the service was unavailable for any reason, your app would still need to know how to startup and run.

If you did create a secure, highly available configuration service, the only change to your application code would be to replace the open call with a call to urllib.urlopen or something similar.

Update, October 31, 2016

The below implementation extends the above ideas.

{
   "common": {
      "authentication": {
         "jwtsecret": "badsecret",
         "jwtexpireoffset": 1800,
         "jwtalgorithm": "HS256",
         "authorize": {
            "role1": "query or group to confirm role1",
            "role2": "query or group to confirm role2"
         },
         "message":{
            "unauthorized":"Access to this system is granted by request. Contact PersonName to get access."
         }
      }
   }
}

The following class encapsulates the management of the configuration file.

import os
import json
 
class authentication_config:
    common = {}
 
    def __init__(self, configuration):
        self.common = configuration['common']['authentication']
 
    def get_config(self):
        return self.common
 
class configuration:
    authentication = None
 
    def __init__(self, config_directory, config_file):
        # load in configuration (directory is assumed relative to this file)
        config_full_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), config_directory, config_file)
        with open(config_full_path, 'r') as myfile:
            configuration_raw = json.load(myfile)
        # prepare authentication
        self.authentication = authentication_config(configuration_raw)
 
 
def main():
    # get path to configuration
    config_directory = 'conf'
    config_file = 'myapp.conf.json'
 
    config = configuration(config_directory, config_file)
    print config.authentication.get_config['jwtsecret']
 
 
if __name__ == '__main__':
    main()

PRV POST

NXT POST