Daniel Watrous on Software Engineering

A Collection of Software Problems and Solutions

Posts tagged Authentication

Software Engineering

Use oAuth to Register Users on My Site using Social Media Credentials

I’m interested in allowing a user to register on my site/app using their social account credentials (e.g. Google, Facebook, LinkedIn, etc.). It should also be possible to register using an email address. Since the site/app will be composed of a handful of microservices, I would want to provide my own identity service, which might includes profile information and roles. This should be possible with oAuth.

I found plenty of examples of how to use oAuth against someone’s social accounts. What I didn’t find were any examples of how to manage user registration and possibly ongoing authentication against a social account. I also didn’t see an examples of how to mix a social oAuth server with an internal oAuth server. The internal oAuth server would provide authentication for each microservice consumed by the site/app. It seemed awkward to keep validating access tokens against the social account oAuth server for each request to local microservices, so this design uses the social access token to get an access token against the internal oAuth server. Here’s how that looks:

social-oauth-register-third-party (You can play with this diagram here)

As you can see, the access token is used to get the initial data to create a user (register) in the internal oAuth server. After registration, the user can still authenticate using their social account, but the account wouldn’t be created a second time. Also notice that the social access token is used to generate the authorization code and eventually the access token for the internal oAuth server, instead of going back to the user for confirmation. In other words, a valid access token from the social oAuth server presumes the user has logged in to authorize access already. The oAuth access token from the internal oAuth server is used to authentication all calls to internal microservices.



Software Engineering

Design principles for REST APIs

I have recently had to work with a few REST APIs that exhibited some poor design choices that I had previously assumed would be obvious. Since they may not be obvious to everyone, I wanted to highlight them.

Idempotent operations

When an operation is idempotent that means that an end state will be identical regardless of how many times the operation is executed. If the end state is dependent on the number of times an operation is executed, then it is not idempotent.

Why is this important?

REST interfaces should assume unreliable networks (which isn’t hard). A consumer of a REST API may issue the same call multiple times if a call fails, or times out or for any other reason. An idempotent operation ensures that the end result of a system is identical in cases where an operation is called multiple times.

The most common violation of this in REST APIs is a call to create a record using a POST. I do agree that in some cases it is unavoidable, but there are many instances where a PUT would be a good replacement. PUT semantics simply make the server representation of a resource match what is sent in. When a POST is unavoidable, it may be possible to add some throttle or comparison function in the API that monitors for replays and either drops them or flags them as potential duplicates.


Authentication is a critical element of any application. Fortunately there are some well established standards which also make your application more interoperable, such as oAuth. Unfortunately, many developers tend to roll their own authentication code, which can be a resource drain and present security risks.

What is this important?

Security is a big deal and has a huge potential impact on business for better or for worse. More and more businesses are getting hacked, which comes with a huge cost to both businesses and consumers.

Accepted standards are more likely to be secure. It is also more likely that there is an existing implementation that can be dropped in to your application.

Personal Information

Personally Identifiable Information (PII), such as username, email, user ID, address, phone, etc. should NEVER be included in a URI. Not only does this muddy the concept that a URI identifies a resource (since a userid in the URI would make it appear to be many resources, one for each userid), but it also compromises personal information. Any necessary personal information should be transmitted in the HEADERS or BODY of the request, not in the URI.

What is this important?

In REST, it’s important to make sure that a given URI identifies one resource, whether that resource is identified as a single item or a collection of items. It must also be reproducible. When personal information is included in a URI, the resource correlation is lost.

Proxy and cache servers often use the URI as the cache key to facilitate lookups. Most web servers also include full URIs in log files. For these reasons, any personal information that is included in the URI may end up persistently stored on any number of systems, including the local browser.

Response Codes

HTTP response codes are standard. Unfortunately many REST APIs make poor use of the standard. Some lump nearly all response states into just one or two response codes, like 500 and 404. Others deviate from the standard and create their own response codes. I’m not going to argue that the available response codes are sufficient for every case. What I will argue is that there is more value in terms of interoperability and adoption when a REST API conforms to existing response codes.

What is this important?

REST APIs are designed to accommodate integration with other systems and applications. Developers of those systems and applications are more likely to be familiar with the standards. It may even be that they are using a library that only accommodates standard response codes (possibly omitting 418 I’m a teapot). Sticking with standard response codes makes integration more straight forward.

Software Engineering

MongoDB Authentication Setup

Authentication in MongoDB provides ‘normal’, which is full read and write, or ‘readonly’ access at a database level. There are two scenarios when authentication comes into play: single server and multi-server. When using a single server, authentication can be enabled but adding --auth to the startup parameters.

When using a replicaset, sharded setup or combination, a key file must be provided and the --keyFile parameter used at startup. This enables each node to communicate with other nodes using a nonce scheme based on the keyFile. In this configuration, --auth is implied and the all MongoDB access then requires authentication.

The details below assume a replicaset configuration.

Creating a keyFile

The keyFile must be 1kB or less. It can be any text and for platform independence, any whitespace is ignored. On Linux, OpenSSL can be used to create a solid keyFile like this:

openssl rand -base64 258 > mongokey

At this point, the local file mongokey will contain something like this:


It’s important to choose a multiple of 3 (e.g. 258) so that not equal signs are added to the keyFile.

Installing keyFile

It’s important to protect this file from unauthorized access. One way to do this is to store it in a way that only the limited user mongod user has access. In my case I moved the file into a directory owned by the mongod user and set permissions restrictively.

mkdir /home/mongod
mv /home/watrous/mongokey /home/mongod/
chown -R mongod.mongod /home/mongod/
chmod -R 700 /home/mongod/

Update MongoDB configuration

With the keyFile in place and secure, I then updated the configuration file, /etc/mongo.conf, to include a reference to the keyFile by adding this line:

keyFile = /home/mongod/mongokey

MongoDB must then be restarted and will load up. After restarting you may notice chatter in the logs about failed authentication. These errors will go away as the same procedure is completed on remaining nodes and they have the keyFile available.

Establishing users

Once a keyFile has been installed as described above, MongoDB then requires authentication. Keep in mind that it is not necessary to add --auth as a startup parameter when using a keyFile.

Complete instructions for adding users can be found here: http://docs.mongodb.org/manual/tutorial/control-access-to-mongodb-with-authentication/.

I began by establishing an admin user. I did this by connecting to mongo locally on the primary node:

[watrous@system ~]$ mongo
MongoDB shell version: 2.0.6
connecting to: test
PRIMARY> use admin
switched to db admin
PRIMARY> db.addUser("admin", "bn%c@4fE0ns$!w4TFao$innIjOBKoPS*")
        "n" : 0,
        "lastOp" : NumberLong("5828987137181089793"),
        "connectionId" : 60,
        "err" : null,
        "ok" : 1
        "user" : "admin",
        "readOnly" : false,
        "pwd" : "f9d7f021d49ccc82b5186d16c664c652",
        "_id" : ObjectId("50e4b8eae0bdfc9063b69c32")
> db.auth("admin", "bn%c@4fE0ns$!w4TFao$innIjOBKoPS*")
PRIMARY> db.system.users.find()
{ "_id" : ObjectId("50e4b8eae0bdfc9063b69c32"), "user" : "admin", "readOnly" : false, "pwd" : "f9d7f021d49ccc82b5186d16c664c652" }

Next I established an account for a specific database. I created two accounts, one with normal access and the other with readonly access.

PRIMARY> use documents
switched to db documents
PRIMARY> db.addUser("documents_full", "*XE@D2x@nc8pfp9iKnA!!Fu!3mTd*HYY")
        "n" : 0,
        "lastOp" : NumberLong("5828988434261213185"),
        "connectionId" : 60,
        "err" : null,
        "ok" : 1
        "user" : "documents_full",
        "readOnly" : false,
        "pwd" : "3cd1cbaec406081b310d7f49b4284c2f",
        "_id" : ObjectId("50e4ba19e0bdfc9063b69c33")
PRIMARY> db.addUser("documents_readonly", "91h#Tv5prInoU%GZQDNF9AoAWN5HTEag", true)
        "n" : 0,
        "lastOp" : NumberLong("5828988696254218241"),
        "connectionId" : 60,
        "err" : null,
        "ok" : 1
        "user" : "documents_readonly",
        "readOnly" : true,
        "pwd" : "87cab9e7ce7a5c731b34b1a0737c2ae9",
        "_id" : ObjectId("50e4ba56e0bdfc9063b69c34")
PRIMARY> db.system.users.find()
{ "_id" : ObjectId("50e4ba19e0bdfc9063b69c33"), "user" : "documents_full", "readOnly" : false, "pwd" : "3cd1cbaec406081b310d7f49b4284c2f" }
{ "_id" : ObjectId("50e4ba56e0bdfc9063b69c34"), "user" : "documents_readonly", "readOnly" : true, "pwd" : "87cab9e7ce7a5c731b34b1a0737c2ae9" }

At this point I was able to verify authentication and access levels.

Software Engineering

MongoDB Secure Mode

Security in MongoDB is relatively young in terms of features and granularity. Interestingly, they indicate that a typical use case would be to use Mongo on a trusted network “much like how one would use, say, memcached.

MongoDB does NOT run in secure mode by default.

As it is, the features that are available are standard, proven and probably sufficient for most use cases. Here’s a quick summary of pros and cons.

  • Pros
    • Nonce-based digest for authentication
    • Security applies across replica set nodes and shard members
  • Cons
    • Few recent replies on security wiki page
    • Course grained access control

User access levels

Course grained access control allows for users to be defined per database and given either read only or read/write access. Since there is no rigid schema in MongoDB, it’s not possible to limit access to a subset of collections or documents.

Limit to expected IPs

Along the lines of the ‘trusted network’ mentioned above, it’s recommended to configure each mongo instance to accept connections from specific ports. For example, you could limit access to the loopback address, or to an IP for a local private network.

Disable http interface

By default, a useful HTTP based interface provides information about the mongodb instance on a machine and links to similar interfaces on related machines in the replica set. This can be disabled by providing –nohttpinterface when starting mongod.

SSL ready

In cases where SSL security is required, Mongo can be compiled to include support for it. The standard downloads do not include this feature. A standard SSL key can be produced in the usual way, using openssl for example.

Software Engineering

Software licensing: Authentication and authorization for admin pages

For simplicity and security I’ve decided to integrate with the Google Account authentication mechanism that’s built into Google App Engine. This allows anyone with a Google account to login to my application without the need to setup another account. This also gives me access to the user’s valid email in order to send messages and other communication related to the service I provide.

So far I have three separate ‘areas’ for interfacing with my service. The first area is comprised of public pages, such as the home page or privacy policy. The next area is the API where RESTful access will take place. That leaves the administration area where an account administrator will be able to view statistics, adjust licenses, etc. These are mapped as follows


The API will require authentication with each call in the form of an apikey (may change to oAuth in the future). I was able to secure the admin area of the site by adding a security-constraint to the web.xml file. Here’s what that looks like.

<web-app ...>

You might have noticed this mechanism is not limited to authentication only. It’s also possible to include authorization preferences by role using role-name.

Software Engineering

Access Apache LDAP authentication details in PHP

Today I was working on a small web application that will run on a corporate intranet. There was an existing LDAP server and many existing web apps use the authentication details cached in the browser (Basic Authentication) to identify a user and determine access levels.

My application is written in PHP and I wanted to leverage this same mechanism to determine the current user and customize my application. Since my searches on Google didn’t pull up anything similar, I want to document what I did.

I did explore the possibility of using PHP’s LDAP library to perform the authentication but decided instead to use the basic authentication provided by Apache. I have two reasons for this. First is that most users are accustomed to this already and developers of other internal applications are familiar with this approach. Second is that authentication details are more easily cached for a long duration, which cuts down on reauthentication. In a trusted intranet environment this is very desirable.

Apache Configuration

To begin with, I installed and configured Apache on Ubuntu Linux. One of the modules that wasn’t enabled by default is authnz_ldap. I did notice that it was available so I ran this command to enable the module:

$ sudo a2enmod authnz_ldap
$ sudo /etc/init.d/apache2 restart

With this module installed I then needed to give it some details about my LDAP server and what paths I wanted protected by LDAP authentication. I accomplished this by adding a <Location> directive to the httpd.conf file. This is what my Location directive looked like:

<Location "/">
  AuthBasicProvider ldap
  AuthType Basic
  AuthzLDAPAuthoritative off
  AuthName "My Application"
  AuthLDAPURL "ldap://directory.example.com:389/DC=example,DC=com?sAMAccountName?sub?(objectClass=*)"
  AuthLDAPBindDN "CN=apache,CN=Users,DC=example,DC=com"
  AuthLDAPBindPassword hackme
  Require valid-user

After making this change another restart was necessary. At this point I reloaded a page that was protected and was able to authenticate as expected.

PHP Access to authentication

This is surprisingly easy (and surprisingly undocumented anywhere that I could find). PHP will automatically populate several $_SERVER superglobal keys with the authentication values cached in Apache. The key values are:


Later I found that you can also add additional values to AuthLDAPURL that will populate additional keys in your $_SERVER superglobal.

At this point you might choose to perform additional operations against the LDAP server using PHP’s library or simply use the available values to customize your intranet web application.