Daniel Watrous on Software Engineering

A Collection of Software Problems and Solutions

Posts tagged wordpress plugin licensing

Software Engineering

Software licensing: A slight change in scope

In the current series of articles I’ve been talking about development of a system to enable premium wordpress plugins to be protected by licenses. While working on the design and going through a brain storming session yesterday with a colleague I had two small realizations.

The first is that from the beginning I have planned to keep the WordPress coupling separate from the main licensing API. In fact, I haven’t planned to include any WordPress specific or shopcart type features in the API. This provides me with some benefits. One is that it keeps the focus of the licensing system on licensing only, not on how or where it will be used. It also makes the API more general and flexible.

The second realization should be rather obvious. This is a software licensing system, not just a WordPress plugin licensing system. I’ll be changing the focus of this series to reflect that more general use case.

Once the plugin system is complete, I’ll then go through a series of specific integrations, including WordPress, PHP, Python, C#, etc.

Even though the titles of the articles will change to software licensing, we’re still working on the same project. We’ve just make the important realization that a more general solution is possible and even desirable.

Software Engineering

WordPress plugin licensing: Wicket on Google App Engine

For the user interface layer of the licensing system I was interested in using a framework like Wicket. It was a lot of work to setup. The process was made somewhat frustrating by the fact that most of Google’s results are a few years old and deal with older versions.

I did finally get it working and here’s the process. Other articles take the default Google App Engine project and modify it to accommodate wicket. I decided to go the other direction and start with the wicket quickstart. This is the process I followed and links to the resources that I used. Hopefully that makes this post age better. We’ll see 🙂

Add Maven support to Eclipse

Since I wanted to start with the current wicket quickstart, I needed maven. Eclipse doesn’t support maven out of the box, but there’s a great plugin that makes working with Maven projects easy. You can get all the details at http://eclipse.org/m2e/.

Generate a wicket quickstart project

The wicket project provides a rather clever mechanism based on Maven to build a quickstart project based on various release version of wicket. You can access the quickstart configuration tool at http://wicket.apache.org/start/quickstart.html.

What that generates is a Maven command. As long as you have Maven installed on your computer, you should be able to generate a quickstart project based on that command. If they maintain this resource then it should be easy to create a fresh project for any future version of wicket. Here’s what I ran to create the quickstart for the licensing system:

Import into Eclipse and Configure for GAE

Now we need to import this project into eclipse and configure it to work with Google App Engine. If you have already install the Maven to Eclipse plugin above then you should be able to import the project. To do this choose File -> Import. From the import dialog choose Existing Maven Project.

On the next screen you provide the path to the directory where the pom.xml file is and choose Finish. Now you have an eclipse project for your wicket quickstart.

Add Google App Engine libraries

Add the Google App Engine libraries to your project by opening the Java Build Path dialog and choosing from available libraries (this assumes you have eclipse setup for google app engine development).

Update project settings for GAE

The process of updating the projects settings feels a bit clumsy because you have to save some settings before others will take. Begin by opening the project settings (right click on project name in Package Explorer panel of Eclipse). The first thing you need to do is set the war directory under Google -> Web Application.

Save this setting and close the properties window. Next you need to reopen the window and navigate to Google -> App Engine. There you will check the box next to “Use Google App Engine”.

This will have generated a problem, but you might not see it. Open the Problems panel in Eclipse using the menu options Window -> Show -> Problems. In the problems view you will see that there’s an error indicating that the appengine-web.xml file is missing. You need to right click on the error and choose Quick Fix. Eclipse will create the appengine-web.xml file for you.

Now would be a good time to make a small update to the appengine-web.xml to enable session support (something that’s disabled by default on the App Engine platform). This change is very straight forward and the line you need to add to the file is already in there as a comment. Here’s the line you need to add:

<sessions-enabled>true</sessions-enabled>

Copy over properties and jars

Now we need a handful of files copied over to our WEB-INF/lib directory so that the GAE development server can find them. This include the followng jar files:

  • log4j
  • slf4j-api
  • slf4j-log4j
  • wicket-core
  • wicket-util
  • wicket-request

I’ve left off the version details since those may be different when you get around to this, but you can see below how your WEB-INF/lib directory should look.

You might have noticed while editing the appengine-web.xml file above that it expects a logging.properties file in WEB-INF. You can create this file with this single line:


.level = WARNING

Update pom.xml output directory, etc.

The default output location for compiled classes is under the ‘target’ directory. The Google App Engine development environment will look under webapp, so we need to modify the Maven configuration file to indicate the directories where we want the compiled classes to go. This is done by adding a couple of elements to the build element as shown below.


<project ...>
  <build>
    <directory>src/main/webapp/WEB-INF</directory>
    <outputDirectory>src/main/webapp/WEB-INF/classes</outputDirectory>
    ...
  </build>
</project>

Related to the above modification, we need to update the classpaths for builds. This is easily done by opening the project properties and removing the hard coded references to the output directory and making sure the default value is correct. Here’s what that looks like:

While we have the pom.xml file open, let’s make a few other small changes. One will reduce the noise of warnings we get during development. This has to do with the encodings of the files that are copied over during a build cycle.


<project ...>
  <properties>
    ...
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>
</project>

Wicketstuff gae-initializer

The final required step is to use the gae-initializer that’s part of the wicketstuff repository. You can view the repository https://github.com/wicketstuff/core.

There are two ways to incorporate this into your project. One is through the Maven configuration file. The other is by including the wicketstuff-gae-initializer-1.5.3.jar directly. In order to make my development environment work correctly I had to include the jar file. You can find the jar files here:

http://repo2.maven.org/maven2/org/wicketstuff/wicketstuff-gae-initializer/1.5.3/

To incorporate the gae-initializer into your pom.xml file, open your pom.xml file and click on the Dependencies tab. You can then Add the dependency with the following details:


<project ...>
  <dependencies>
    <dependency>
      <groupId>org.wicketstuff</groupId>
      <artifactId>wicketstuff-gae-initializer</artifactId>
      <version>YOUR-WICKET-VERSION</version>
    </dependency>
    ...
  </dependencies>
</project>

Here’s what it looks like in Eclipse:

I suppose that alone should be enough, and it might be enough for your deployment to production in Google App Engine, but I found that I also need to add the JAR to my build path as an external JAR. This is how I did that:

I now have a running wicket application in Google App Engine.

Resources

Below are a list of old (and very old) tutorials. I couldn’t get these to work with the current version of wicket and GAE. Still, they may be useful if you encounter problems related to features that I didn’t use or test.
http://www.danwalmsley.com/2009/04/08/apache-wicket-on-google-app-engine-for-java/
http://stronglytypedblog.blogspot.com/2009/04/wicket-on-google-app-engine.html
http://stronglytypedblog.blogspot.com/2009/07/wicket-spring-jdo-on-google-app-engine.html

Documentation for gae-initializer in wicketstuff core repository:
https://github.com/wicketstuff/core/wiki/Google-AppEngine-Initializer

This thread covered some points that might fill in gaps:
http://mail-archives.apache.org/mod_mbox/wicket-users/201003.mbox/%3C1269607382.11633.11.camel@kaffeeserver%3E

This article had some discussion of serialization problems. I think that the gae-initializer that I cover in this article addresses the points in the article below.
http://thoughts.inphina.com/2010/10/20/managing-wicket-serialization-problem-on-google-app-engine/

Complaints about GAE not supporting wicket properly (related to above article):
http://code.google.com/p/googleappengine/issues/detail?id=2500

My most recent thread on the wicket users list working through some of the points in this article:
http://apache-wicket.1842946.n4.nabble.com/Wicket-on-Google-App-Engine-td4259205.html

Software Engineering

WordPress plugin licensing: API and class diagram

In the first article in this series, I sketched out some of the basic design ideas for the wordpress plugin licensing system.

The next step is to come up with some more concrete details about how the application will work, what information it will manage and what the interfaces will look like when accessing it. I really like using the UML as I flesh out a design. In particular I find the class diagram and activity diagram useful.

I frequently make use of stereotypes to extend the meaning of my classes. I’m not sure if this is exactly how the UML was intended to be used, but it’s really helpful. At times I also use simple tables, hand drawings, etc. if it helps capture an important point.

Here are the first design documents showing the classes I have in mind for the application and the RESTful API that will provide access to them.

Classes and API

Design choices

My initial design included normalized classes for Licensee, Product and LicenseTemplate. This would prevent duplication of licensee data across multiple licenses and would facilitate reporting by liensee and product. I also had APICall associated with AdministratorAccount since each API call could correspond to various objects in the system.

As I fleshed out the API, a few drawbacks came to mind regarding the normalization I mentioned above. First was the additional complexity in managing all of the normalized classes and associating parent and child objects. Since I am not targeting an RDBMS, the traditional SQL joins and other query approaches don’t benefit me with properly normalized design. I can make the application much more performant and simple by keeping everything in a single SiteLicenses container as shown.

Next design steps

You can see from the diagram above that I haven’t begun assigning responsibilities yet. However, it’s easy to see already where some of the key responsibilities, like isPremiumAccount(), getSiteLicenses(), revokeLicense(), getAPICallsMinutes(Integer minutesBack), etc. will end up.

Software Engineering

WordPress plugin licensing: Choose a Java framework

How frameworks came to be

Engineers have a few potentially conflicting tendencies that can become very pronounced in the software they write. One tendency is that we’re empire builders. In other words, we build, enhance, improve and in other ways expand the scope of our software solutions over time. Not to mention that we want our solution to be accepted, embraced and adored as the best possible solution for the problem we set out to solve. The result of this tendency can be seen in the ever growing, ever more complex frameworks available today.

The other tendency is to be overly critical of the software solutions created by other engineers. Sometimes this critical nature comes out as mild concerns related to difficult to test aspects like scalability. At other times it may come out as fundamental objections to the framework architecture. However severe the criticism, many engineers decide to start fresh and build something entirely new that fits their style. The result of this can be seen in the large variety of frameworks available to solve very similar problems.

Two projects that illustrate the points above are the Spring framework based on Rod Johnson’s work, and Google’s Guice framework. Both embrace dependency injection (inversion of control), but they disagree on implementation. While Spring has become an empire, Guice claims to want to remain small and lightweight.

It is very interesting that the tendencies I’ve just discussed have been the subject of academic research. Recently, the Harvard Business School published a paper they entitled “The IKEA effect: When labor leads to love”, referring to the furniture retailer. They found that “When people construct products themselves, from bookshelves to Build-a-Bears, they come to overvalue their (often poorly made) creations”.

You can read this research yourself: Harvard Business Review summary of IKEA effect or ready the full IKEA Effect paper.

I’m not suggesting that frameworks are poorly made. On the contrary, some of them are very mature, show great insight and can save the developer many hours of work and greatly improve the end results of his efforts. What does concern me at times is the quasi-religious adoration some developers afford a particular framework. Before becoming a satellite to a framework, a developer should ask whether it’s a fit for the problem he has in front of him.

Much of the same reasoning applies to discussion of programming languages. Expertise in one language should not rule out other language possibilities if a new problem justifies a new approach.

Do I have to use a framework?

One might ask if it is necessary at all to use a framework. Martin Fowler has encouraged the use of plain old java objects (POJO) for business logic. Rod Johnson’s efforts have the objective of reducing coupling with a framework.

The purpose of the framework is to provide an abstraction for lower level libraries. This often includes scaffolding necessary to perform some common or frequent tasks. Everything that you can do in the framework can also be done directly in plain Java code, but the framework may save you time.

For example, Hibernate may make access to a datastore more object friendly. Spring may make integration more descriptive. GWT can both accelerate the development life cycle for rich web clients and at the same time reduce risk in deployment with Java compile time checks and cross browser issues taken care of.

The complexity of the problem I’m trying to solve will often determine whether or not I use a framework, and which framework might be a fit. If my application will have very little javascript then I probably don’t need GWT (or even jQuery). If there will be very few interface pages and session management is minimal then I probably don’t need Wicket. There are cases when jsp pages with simple includes are more than adequate and even more appropriate.

More on frameworks

I really liked this presentation I came across while looking into frameworks http://www.slideshare.net/mraible/comparing-flex-grails-gwt-seam-struts-2-and-wicket-presentation

Framework choice for WordPress plugin licensing

The focus of the current project is to create an easy to use RESTful API that will enable license generation and verification. That means there is very little Javascript and so GWT seems a poor fit. At this stage I’ve also eliminated Seam and Flex for Java.

Since Google App Engine for Java is the target for initial deployment, I did a review of many of the popular remaining popular frameworks available to see how well they play with GAE. In some cases, this might be a good time to reevaluate the choice of GAE and possibly lean toward Amazon EC2 if there were a compelling reason to choose a framework that didn’t perform as well on GAE.

The most interesting frameworks are wicket and Spring, but due to the low complexity and native support the first iteration of my application will use basic servlets and jsp.

Software Engineering

WordPress plugin licensing: Setup Eclipse for Google App Engine development

Having chosen Google App Engine and Java as the target for initial development it’s time to setup a development environment. Google provides a plugin to Eclipse that makes this easy. They do a great job explaining how to setup the environment at this URL:

http://code.google.com/appengine/docs/java/tools/eclipse.html

Setup problems

One hang up I had was an error when first trying to install the plugin in eclipse. It complained about “An error occurred while collecting items to be installed”. Several files came back saying “Unable to read repository” and “Read timed out”. It was possible to download these directly (outside of Eclipse) so I downloaded the ones that produced errors and moved then into the plugins folder. After that the installation process worked as outlined in the link above.

Possibly related to the problem I just mentioned was that I had to manually extract com.google.appengine.eclipse.sdkbundle_1.6.1 into its own directoy and in preferences I had to point to the sdk manually. I did this by going to Window -> Preferences. In the preferences window navigate to Google -> App Engine. There you can Add a link to the directory where you put your sdk files from the above mentioned jar.

In case you plan to develop for GWT, you may need to set additional variables for your Java runtime environment

Using Eclipse for GAE Development

About halfway down the page for installing the Eclipse plugin (link above) there are two sections that take you through the process of creating and running a project. When I created a new project I selected the checkbox to generate sample project code. This created a Hello World project and made it a snap to test that my environment was setup correctly.

If you’ve accomplished everything above then you should be able to load http://localhost:8888 and see your test application.

One thing I love about developing for app engine in Java with this setup is that the build/deploy process happens automatically. Eclipse rebuilds classes each time the source for that class is saved. The newly compiled class is inserted into the running web server. The same is true for changes to static files and resources. This means you can simply save and refresh, much like you would when using an interpreted language. A small gotcha is that you must ‘Debug As’, NOT ‘Run As’ in order for this dynamic refresh behavior to work.

Software Engineering

WordPress plugin licensing: Google App Engine vs. Amazon EC2

In the introduction to this series, I outlined some of the requirements for the WordPress plugin licensing platform: Speed, reliability and scalability. These are critical. Just imagine what would happen if any of those were missing.

Requirements Justification

A slow platform might result in significantly fewer sales. One of our use cases is to provide a free, limited time trial, and poor performance when installing or using a plugin would almost certainly decrease sales conversions. Reliability issues would, at a minimum, reduce developer confidence when coupling a new plugin to the licensing platform. Finally, if the speed and reliability don’t scale then the market of potential consumers is limited to smaller plugins.

Possible solutions

Fortunately the problem of speed, reliability and scalability have already been solved. I know that it’s possible to build out servers, load balance them and otherwise build out systems to achieve these three aims, but I have something much simpler in mind. The two most compelling options available today both allow a developer to leverage the infrastructure of very large companies that exist solely on the internet: Amazon and Google.

The business model of both Amazon and Google require them to build out their own internal infrastructure to accommodate peak volume. The big downside to this is that the majority of the time, some or most of that infrastructure is sitting idle. A somewhat interesting upside to the scale of their infrastructure is that they have had to develop internal processes that enable them to expand supply in step with demand. In other words, they have to be able to add additional resources on the fly in the event of a new record peak. That may not sound as impressive as it is 🙂

At some point, each of these companies realized that they could leverage their unused infrastructure to increase their revenue. They more or less sub-lease existing resources to third parties. As this product model developed they may have isolated the resources they sell from the resources that power their main websites, but there is still a great deal of play between them. The two offerings are Google App Engine (GAE) and Amazon Web Services (AWS).

App Engine vs. Amazon Web Services

.

There are many more differences between these two platforms than I have time to get into here. However, one distinction between the two is helpful. Amazon offers a wide range of services (they add new services often) that provide the developer with a great deal of flexibility. However, the burden of choosing the right platform components and interconnecting them is also on the developer.

Google on the other hand has a more narrowly defined and inclusive platform. Rather than separating content distribution, processing, messaging, etc., Google keeps it all under the same hood. This reduces complexity for the developer at the cost of some flexibility.

This distinction is rather natural when you consider the diversity of products and engagement channels employed by Amazon and compare that to the more narrow range of services and engagement channels employed by Google.

The winner?

For the licensing project that I’m developing in this series, the scope is well defined and not overly complex. Google App Engine is the most appealing due to the ease of working in a local development environment and the ability to deploy and test on the live platform under the free quota limits (no initial cost or setup). It’s important to note that choosing Google’s platform instead of Amazon’s doesn’t make Amazon the loser and it doesn’t have to mean that I need to exclusively run on Google’s platform forever.

GAE provides both Python and Java enviornments. If I choose Java and approach the design carefully (e.g. good datastore abstraction…), it may not require too much effort to deploy on an Amazon EC2 instance if that becomes more appealing down the road.

Conclusion

The WordPress plugin licensing system will target the Google App Engine platform initially. Special attention will be given to abstracting the datastore so that I can take advantage of Google’s fast and scalable datastore and leave myself flexibility to move to an alternate if I deploy on Amazon’s platform in the future. Java is a first class citizen on both platforms. This provides some assurance that mainstream, mature frameworks will run smoothly. It also typically means that there will be plenty of documentation and support to accelerate development and deployment along.