Daniel Watrous on Software Engineering

A Collection of Software Problems and Solutions

Posts tagged flask

Software Engineering

Configuration of python web applications

Hopefully it’s obvious that separating configuration from application code is always a good idea. One simple and effective way I’ve found to do this in my python (think bottle, flask, etc.) apps is with a simple JSON configuration file. Choosing JSON makes sense for a few reasons:

  • Easy to read (for humans)
  • Easy to consume (by your application
  • Can be version alongside application code
  • Can be turned into a configuration REST service

Here’s a short example of how to do this for a simple python application that uses MongoDB. First the configuration file.

    "use_ssl": false,
    "authentication": {
        "enabled": false,
        "username": "myuser",
        "password": "secret"},
    "mongodb_port": 27017,
    "mongodb_host": "localhost",
    "mongodb_dbname": "appname"

Now in the python application you would load the simple JSON file above in a single line:

# load in configuration
configuration = json.load(open('softwarelicense.conf.json', 'r'))

Now in your connection code, you would reference the configuration object.

# establish connection pool and a handle to the database
mongo_client = MongoClient(configuration['mongodb_host'], configuration['mongodb_port'], ssl=configuration['use_ssl'])
db = mongo_client[configuration['mongodb_dbname']]
if configuration['authentication']['enabled']:
    db.authenticate(configuration['authentication']['username'], configuration['authentication']['password'])
# get handle to collections in MongoDB
mycollection = db.mycollection

This keeps the magic out of your application and makes it easy to change your configuration without direct changes to your application code.

Configuration as a web service

Building a web service to return configuration data is more complicated for a few reasons. One is that you want to make sure it’s secure, since most configuration data involves credentials, secrets and details about deployment details (e.g. paths). If a hacker managed to get your configuration data that could significantly increase his attack surface. Another reason is that it’s another potential point of failure. If the service was unavailable for any reason, your app would still need to know how to startup and run.

If you did create a secure, highly available configuration service, the only change to your application code would be to replace the open call with a call to urllib.urlopen or something similar.

Update, October 31, 2016

The below implementation extends the above ideas.

   "common": {
      "authentication": {
         "jwtsecret": "badsecret",
         "jwtexpireoffset": 1800,
         "jwtalgorithm": "HS256",
         "authorize": {
            "role1": "query or group to confirm role1",
            "role2": "query or group to confirm role2"
            "unauthorized":"Access to this system is granted by request. Contact PersonName to get access."

The following class encapsulates the management of the configuration file.

import os
import json
class authentication_config:
    common = {}
    def __init__(self, configuration):
        self.common = configuration['common']['authentication']
    def get_config(self):
        return self.common
class configuration:
    authentication = None
    def __init__(self, config_directory, config_file):
        # load in configuration (directory is assumed relative to this file)
        config_full_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), config_directory, config_file)
        with open(config_full_path, 'r') as myfile:
            configuration_raw = json.load(myfile)
        # prepare authentication
        self.authentication = authentication_config(configuration_raw)
def main():
    # get path to configuration
    config_directory = 'conf'
    config_file = 'myapp.conf.json'
    config = configuration(config_directory, config_file)
    print config.authentication.get_config['jwtsecret']
if __name__ == '__main__':
Software Engineering

Revisit Webfaction Hosting

Several years ago I hosted with webfaction for about a year. I was drawn to them at the time because they allowed SSH access and I could run Java applications on my account. Those were not common features available under shared hosting at the time. I didn’t end up deploying any Java applications and the PHP sites I did deploy routed through webfaction’s nginx to PHP configuration which frequently failed. That meant that many visitors to my site saw nginx errors rather than my web page. When they couldn’t resolve the issue I moved my hosting to hostgator and have been extremely happy ever since.

I recently decided to explore Python Flask with MongoDB and was looking at hosting options. Google App Engine is a little too restrictive and I’ve had a mixed experience while working on the Software Licensing system that I’ve developed for that platform. I considered several VPS options including Amazon EC2 and Linode. As I was looking around I thought about webfaction again.

Ready to run Flask and MongoDB

I did a little searching and found that I could deploy Flask applications easily on webfaction. I could also deploy a MongoDB instance on webfaction.

I decided to give them another try and paid my first month’s fee. With about an hour of setup I had successfully deployed a Python Flask “Hello World!” application and had a running MongoDB instance. It was surprisingly easy, especially considering that with any VPS solution I would have needed to setup and worry about a lot more than just my Flask application. It saved me time and money.

Caveat Emptor

What I don’t know is whether they have addressed nginx errors (I recall that they were 502 Bad Gateway errors). Apparently it’s related to the nginx server not getting a suitable reply from the application. If I find myself fighting with that again this time around I may end up on a VPS anyway, but for development, it’s hard to beat their pricing and flexibility.

I really hope it works out that I can run in production on webfaction. I’ll keep you posted.