Daniel Watrous on Software Engineering

A Collection of Software Problems and Solutions

Posts tagged servlet

Software Engineering

Wicket + Guice including unittests

Last week I spent way too much time integrating Apache Wicket and Google Guice. Yikes! The most difficult part for me was getting the initialization to happen in the right order. A big Thank You to Dan Retzlaff on the Wicket list for helping work through these details.

The details below were applied to a Wicket quickstart project for Wicket 6.0.0.

Design Decisions

It was important to me to keep the application tier separate from web tier. I actually maintain each in a separate repository. I have several motivations for this, such as:

  • Clean separation of concerns. In other words, prevent logic from ending up in my Wicket pages
  • Independent revisions and release cycles between web tier and application tier
  • Easier to divide work between scrum teams

I include the application tier into the Wicket front end as a jar. By the way, this also makes it easy to include my application tier into a Jersey REST interface and other legacy servlets.

It was also important to maintain a mock package for fast unittests. Since the data providers are managed in the application tier, the mock package lives there, further enforcing the separation of concerns.

Implementation Approach

After some experimentation I decided to use the GuiceServlet approach. I started by adding the following maven dependencies to my pom.xml for the Wicket quickstart.

        <dependency>
            <groupId>com.google.inject</groupId>
            <artifactId>guice</artifactId>
            <version>3.0</version>
        </dependency>
        <dependency>
            <groupId>com.google.inject.extensions</groupId>
            <artifactId>guice-servlet</artifactId>
            <version>3.0</version>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>org.apache.wicket</groupId>
            <artifactId>wicket-guice</artifactId>
            <version>6.7.0</version>
            <type>jar</type>
        </dependency>

My web.xml defines only the GuiceFilter and a listener to initialize the injector when the servlet context is created.

<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
	version="2.5">
 
	<display-name>guicewickettest</display-name>
 
        <listener>
            <listener-class>com.danielwatrous.myapp.web.MyGuiceServletConfig</listener-class>
        </listener>
 
        <filter>
            <filter-name>guiceFilter</filter-name>
            <filter-class>com.google.inject.servlet.GuiceFilter</filter-class>
        </filter>
 
        <filter-mapping>
            <filter-name>guiceFilter</filter-name>
            <url-pattern>/*</url-pattern>
        </filter-mapping>
</web-app>

As you’ll see, this keeps configuration details in code, rather than XML. The web.xml remains simple. That brings us to the GuiceServletConfig, which is where we initialize the injector.

You may recall that listeners receive event notifications at well defined points in the life cycle of a web application or session. Here’s the MyGuiceServletConfig that’s referenced in web.xml:

package com.danielwatrous.myapp.web;
 
import com.google.inject.Guice;
import com.google.inject.Injector;
import com.google.inject.servlet.GuiceServletContextListener;
import com.danielwatrous.myapp.modules.MongoMyappModule;
 
public class MyGuiceServletConfig extends GuiceServletContextListener {
 
    @Override
    protected Injector getInjector() {
        return Guice.createInjector(new MyappServletModule(), new MongoMyappModule());
    }
 
}

The GuiceServletContextListener implements ServletContextListener, which ends up executing when the application context is created (or destroyed). This way we have the injector available before executing any application code.

Another thing you may notice is that I create the injector with two modules, not just one. The first module is the ServletModule, and I’ll show that to you in a minute. The next module is the application tier module that has been included as a jar in the Wicket application. The single injector available throughout the Wicket application will be able to inject servlet/Wicket related components in addition to application tier components.

Let’s have a closer look at MyappServletModule:

package com.danielwatrous.myapp.web;
 
import com.google.inject.Inject;
import com.google.inject.Provider;
import com.google.inject.Singleton;
import com.google.inject.servlet.ServletModule;
import java.util.HashMap;
import java.util.Map;
import org.apache.wicket.protocol.http.IWebApplicationFactory;
import org.apache.wicket.protocol.http.WebApplication;
import org.apache.wicket.protocol.http.WicketFilter;
 
public class MyappServletModule extends ServletModule {
    @Override
    protected void configureServlets() {
        filter("/*").through(WicketFilter.class, createWicketFilterInitParams());
        bind(WebApplication.class).to(WicketApplication.class);
        bind(WicketFilter.class).to(CustomWicketFilter.class).in(Scopes.SINGLETON);
    }
 
    @Singleton
    private static class CustomWicketFilter extends WicketFilter {
 
        @Inject
        private Provider<WebApplication> webApplicationProvider;
 
        @Override
        protected IWebApplicationFactory getApplicationFactory() {
            return new IWebApplicationFactory() {
                @Override
                public WebApplication createApplication(WicketFilter filter) {
                    return webApplicationProvider.get();
                }
 
                @Override
                public void destroy(WicketFilter filter) {
                }
            };
        }
    }
 
    private Map<String, String> createWicketFilterInitParams() {
        Map<String, String> wicketFilterParams = new HashMap<String, String>();
        wicketFilterParams.put(WicketFilter.FILTER_MAPPING_PARAM, "/*");
        wicketFilterParams.put("applicationClassName", "com.danielwatrous.myapp.web.WicketApplication");
        return wicketFilterParams;
    }
}

You may notice that I added a mechanism to provide WicketFilter with additional parameters. Next I bind my WebApplication and WicketFilter classes to specific implementations. The CustomWicketFilter overrides the typical behavior of the WicketFilter which usually takes a string reference to the WebApplication class. Instead it now uses the injected WebApplication object.

As you’ll see below, this step of injecting the desired WebApplication is critical to enabling unittests, primarily because it allows us to construct the WebApplication with an injector.

package com.danielwatrous.myapp.web;
 
import com.google.inject.Inject;
import com.google.inject.Injector;
import org.apache.wicket.guice.GuiceComponentInjector;
import org.apache.wicket.protocol.http.WebApplication;
 
public class WicketApplication extends WebApplication {    	
    private final Injector injector;
 
    @Inject
    public WicketApplication(Injector injector) {
        this.injector = injector;
    }
 
    @Override
    public Class<HomePage> getHomePage() {
        return HomePage.class;
    }
 
    @Override
    public void init() {
        super.init();
        getComponentInstantiationListeners().add(new GuiceComponentInjector(this, injector));
    }
}

At this point Wicket and Guice are successfully integrated. Let’s have a look at what needs to happen to make the unittests work.

Unittests

The only real change that’s required to make the unittests work is in the setUp function of the unittest. Since the WebApplication above was modified to receive an injector, all we need to do is create an injector and provide it at instantiation.

package com.danielwatrous.myapp;
 
import com.google.inject.Guice;
import com.google.inject.Injector;
import com.danielwatrous.myapp.modules.TestMyappModule;
import com.danielwatrous.myapp.web.HomePage;
import com.danielwatrous.myapp.web.WicketApplication;
import org.apache.wicket.util.tester.WicketTester;
import org.junit.Before;
import org.junit.Test;
 
public class TestHomePage {
 
    private WicketTester tester;
 
    @Before
    public void setUp() {
        Injector injector = Guice.createInjector(new TestMyappModule());
        tester = new WicketTester(new WicketApplication(injector));
    }
 
    @Test
    public void homepageRendersSuccessfully() {
        //start and render the test page
        tester.startPage(HomePage.class);
 
        //assert rendered page class
        tester.assertRenderedPage(HomePage.class);
    }
}

You’ll notice in this case I create an injector that doesn’t include the ServletModule and uses the TestMyappModule. Since the Wicket unittests don’t operate within a full web context I don’t need the ServletModule. Additionally my TestMayappModule makes use of my mock package. This allows the tests to run without access to any external resources. It also keeps the tests very fast!

Wicket Pages

Accessing the injector in your Wicket pages is easy. All you need to do is inject it. Here’s how that looks:

package com.danielwatrous.myapp.web;
 
import com.google.inject.Inject;
import com.google.inject.Injector;
import com.danielwatrous.myapp.domain.QuickLink;
import org.apache.wicket.markup.html.WebPage;
import org.apache.wicket.markup.html.basic.Label;
import org.apache.wicket.request.mapper.parameter.PageParameters;
 
public class HomePage extends WebPage {
    private static final long serialVersionUID = 1L;
    @Inject private Injector injector;
 
    public HomePage(final PageParameters parameters) {
	super(parameters);
 
	add(new Label("version", getApplication().getFrameworkSettings().getVersion()));
 
        // TODO Add your page's components here
        QuickLink quickLink = injector.getInstance(QuickLink.class);
	add(new Label("quickLink ", quickLink.buildQuickLink()));
    }
}

Great Combination

After working through the particulars, this implementation feels clean and flexible. Configuration is in the code and benefits from compile time type checking. Unittests are working and fast with very little modification to the traditional Wicket approach, which keeps the application testable.

Resources:

http://apache-wicket.1842946.n4.nabble.com/Wicket-Guice-unittests-td4652853.html
https://gist.github.com/3880246

Software Engineering

RESTful Java Servlet: Serializing to/from JSON with Jackson

In a previous article I demonstrated one way to create a RESTful interface using a plain Java Servlet. In this article I wanted to extend that to include JSON serialization using Jackson.

I found a very simple article showing a basic case mapping a POJO to JSON and back again. However, when I copied this straight over I got the following error:

org.codehaus.jackson.map.JsonMappingException: No serializer found for class DataClass and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationConfig.Feature.FAIL_ON_EMPTY_BEANS)

I found there were two ways to get past that error. The first was to use the Jackson annotations to define properties more directly. The other was to add getters and setters to the class. Here is my DataClass with both configurations.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import java.util.ArrayList;
import java.util.List;
 
import org.codehaus.jackson.annotate.JsonAutoDetect;
import org.codehaus.jackson.annotate.JsonIgnoreProperties;
import org.codehaus.jackson.annotate.JsonProperty;
 
@JsonAutoDetect   // use this annotation if you don't have getters and setters for each JsonProperty
//@JsonIgnoreProperties(ignoreUnknown = true)    // use this if there isn't an exact correlation between JSON and class properties
public class DataClass {
 
  //@JsonProperty("theid") // use this if the property in JSON has a different identifier than your class
  @JsonProperty
  private int id = 1;
 
  @JsonProperty
  private String name = "Test Class";
 
  @JsonProperty
  private List<String> messages = new ArrayList<String>() {
    {
      add("msg 1");
      add("msg 2");
      add("msg 3");
    }
  };
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
import java.util.ArrayList;
import java.util.List;
 
public class DataClass {
 
  private int id = 1;
 
  private String name = "Test Class";
 
  private List<String> messages = new ArrayList<String>() {
    {
      add("msg 1");
      add("msg 2");
      add("msg 3");
    }
  };
 
  public String toString() {
    return "User [id=" + id + ", name=" + name + ", " + "messages=" + messages + "]";
  }
  public int getId() {
    return id;
  }
  public void setId(int id) {
    this.id = id;
  }
  public String getName() {
    return name;
  }
  public void setName(String name) {
    this.name = name;
  }
  public List<String> getMessages() {
    return messages;
  }
  public void setMessages(List<String> messages) {
    this.messages = messages;
  }
}

We are now free to modify the doGet and doPost methods in our original servlet to serialize and deserialize the DataClass to and from JSON. Here’s the code for that:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
...
  protected void doGet(HttpServletRequest request, HttpServletResponse response)
      throws ServletException, IOException {
    PrintWriter out = response.getWriter();
 
    DataClass mydata = new DataClass();
    ObjectMapper mapper = new ObjectMapper();
 
    try {
      // display to console
      out.println(mapper.writeValueAsString(mydata));
    } catch (JsonGenerationException e) {
      e.printStackTrace();
    } catch (JsonMappingException e) {
      e.printStackTrace();
    } catch (IOException e) {
      e.printStackTrace();
    }
    out.close();
  }
 
  protected void doPost(HttpServletRequest request, HttpServletResponse response)
      throws ServletException, IOException {
    PrintWriter out = response.getWriter();
 
    ObjectMapper mapper = new ObjectMapper();
 
    try {
      // read from file, convert it to user class
      DataClass user = mapper.readValue(request.getReader(), DataClass.class);
      // display to console
      out.println(user);
    } catch (JsonGenerationException e) {
      e.printStackTrace();
    } catch (JsonMappingException e) {
      e.printStackTrace();
    } catch (IOException e) {
      e.printStackTrace();
    }
    out.close();
  }
...

Now you can easily serialize data to and from JSON using Jackson and POJOs without the need for a mapping file. There are even convenient annotations available that allow you to accommodate differences between the JSON and POJO properties.

Software Engineering

RESTful Java Servlet

For a recent project I found that a RESTful interface would be appropriate. My first inclination was to use Jersey (or one of the JAX-RS implementations available). The environment where this new REST API would deploy is still using Java 1.5. This became a major roadblock when I was found that none of the JAX-RS implementations provide support for the Java 1.5 virtual machine. This is not surprising since it’s few YEARS past EOSL (end of support life) for Java 1.5, but disappointing still the same.

After spending a day or so with the available frameworks, trying to get one of them to work in Java 1.5, I grew nervous that even if I did succeed, that puts my implementation squarely in a non-support region, even on the mailing lists. So I decided to see how close I could come to a simple, reliable RESTful interface using a plain old Java servlet (POJS?).

While I’m disappointed that I’m unable to leverage an existing, mature framework, I was happy with the outcome. Here are the details

HTTP Methods built in

To begin with, the servlet specification defines methods for the HTTP actions: GET, POST, PUT, DELETE, OPTIONS and HEAD.

Each of these methods receives a HttpServletRequest and HttpServletResponse object, which makes it easy to access the payload of the request and construct a response. Manipulation of return headers, payload content, response codes (200, 404, etc.) are all available directly through the HttpServletResponse object.

The only imports required to accomplish this were available in the standard Java environment, which relieved me of the need to worry about interoperability.

1
2
3
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

URI mapping

The next challenge was how to map RESTful URIs on to the methods in my servlet. At first glance, the conventional mapping done in the web.xml file wouldn’t be sufficient. The problem was with the extra parameters that were embedded in the URI. This goes just beyond what the mapping delineates in web.xml.

What I found was that a wildcard mapping and use of HttpServletRequest.getPathInfo() gave me all the information that I needed to complete the mapping. This is what my web.xml file looks like

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<?xml version="1.0" encoding="UTF-8"?>
<web-app ...>
	<display-name>testrestservlet</display-name>
	<servlet>
		<description></description>
		<display-name>TestRestServlet</display-name>
		<servlet-name>TestRestServlet</servlet-name>
		<servlet-class>com.danielwatrous.testrestservlet.TestRestServlet</servlet-class>
	</servlet>
	<servlet-mapping>
		<servlet-name>TestRestServlet</servlet-name>
		<url-pattern>/api/v1/*</url-pattern>
	</servlet-mapping>
</web-app>

The call I mentioned above, getPathInfo(), returns a string with everything that is in place of the asterisk (*) in my url-pattern above. This means that a URI of the form http://server/testrestservlet/api/vi/resource/id would return a string of “/resource/id” when getPathInfo() is called.

Avoid String Handling

I wanted to avoid messy string manipulation or analysis to parse out the details of this resource identification information. This ruled out any string splitting and checking for indexOf or startsWith.

Instead I wanted to be deliberate and reduce the possibility that a mis-mapping would slip through or that a future developer would misunderstand the exact nature of what to expect. For this reason I chose to use regular expressions. This provides a clear pattern of what my URI should look like, and it will be obvious to other developers. It also reduces the chances of a bad URI resulting in an inconsistent result.

I always use Kodos to develop regular expressions. After I got the regular expressions worked out for each class, I created an inner class inside the servlet that would help me map the URI to a specific request and give me access to the parameters embedded in the request URI. Here’s what that inner class looks like.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
  private class RestRequest {
    // Accommodate two requests, one for all resources, another for a specific resource
    private Pattern regExAllPattern = Pattern.compile("/resource");
    private Pattern regExIdPattern = Pattern.compile("/resource/([0-9]*)");
 
    private Integer id;
 
    public RestRequest(String pathInfo) throws ServletException {
      // regex parse pathInfo
      Matcher matcher;
 
      // Check for ID case first, since the All pattern would also match
      matcher = regExIdPattern.matcher(pathInfo);
      if (matcher.find()) {
        id = Integer.parseInt(matcher.group(1));
        return;
      }
 
      matcher = regExAllPattern.matcher(pathInfo);
      if (matcher.find()) return;
 
      throw new ServletException("Invalid URI");
    }
 
    public Integer getId() {
      return id;
    }
 
    public void setId(Integer id) {
      this.id = id;
    }
  }

Now I’m set to override the methods for each of my HTTP actions.

The Servlet

At this point, creating the servlet is trivial. The mapping provides all path (resource) information to the servlet and our inner class is responsible to determining whether the URI makes sense for our application and to make that information available in a sensible way for processing of the request in the action methods. Here’s what I came up with:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
package com.danielwatrous.testrestservlet;
 
import java.io.IOException;
import java.io.PrintWriter;
 
import java.util.regex.Pattern;
import java.util.regex.Matcher;
 
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
 
public class TestRestServlet extends HttpServlet {
 
  private class RestRequest {
    // Accommodate two requests, one for all resources, another for a specific resource
    private Pattern regExAllPattern = Pattern.compile("/resource");
    private Pattern regExIdPattern = Pattern.compile("/resource/([0-9]*)");
 
    private Integer id;
 
    public RestRequest(String pathInfo) throws ServletException {
      // regex parse pathInfo
      Matcher matcher;
 
      // Check for ID case first, since the All pattern would also match
      matcher = regExIdPattern.matcher(pathInfo);
      if (matcher.find()) {
        id = Integer.parseInt(matcher.group(1));
        return;
      }
 
      matcher = regExAllPattern.matcher(pathInfo);
      if (matcher.find()) return;
 
      throw new ServletException("Invalid URI");
    }
 
    public Integer getId() {
      return id;
    }
 
    public void setId(Integer id) {
      this.id = id;
    }
  }
 
  protected void doGet(HttpServletRequest request, HttpServletResponse response)
      throws ServletException, IOException {
    PrintWriter out = response.getWriter();
 
    out.println("GET request handling");
    out.println(request.getPathInfo());
    out.println(request.getParameterMap());
    try {
      RestRequest resourceValues = new RestRequest(request.getPathInfo());
      out.println(resourceValues.getId());
    } catch (ServletException e) {
      response.setStatus(400);
      response.resetBuffer();
      e.printStackTrace();
      out.println(e.toString());
    }
    out.close();
  }
 
  // implement remaining HTTP actions here
  ... 
 
}

Conclusion

This turned out to be a clear enough way to capture REST like URIs and map those onto a servlet for processing. Regular expressions provide for reliable and clear access to resource details and should be easy for future developers to quickly understand and extend.

In my next article, I’ll show you how I extend this servlet using the Jackson JSON library and a helper object to serialize and deserialize JSON payloads in RESTful requests.

Software Engineering

WordPress plugin licensing: Choose a Java framework

How frameworks came to be

Engineers have a few potentially conflicting tendencies that can become very pronounced in the software they write. One tendency is that we’re empire builders. In other words, we build, enhance, improve and in other ways expand the scope of our software solutions over time. Not to mention that we want our solution to be accepted, embraced and adored as the best possible solution for the problem we set out to solve. The result of this tendency can be seen in the ever growing, ever more complex frameworks available today.

The other tendency is to be overly critical of the software solutions created by other engineers. Sometimes this critical nature comes out as mild concerns related to difficult to test aspects like scalability. At other times it may come out as fundamental objections to the framework architecture. However severe the criticism, many engineers decide to start fresh and build something entirely new that fits their style. The result of this can be seen in the large variety of frameworks available to solve very similar problems.

Two projects that illustrate the points above are the Spring framework based on Rod Johnson’s work, and Google’s Guice framework. Both embrace dependency injection (inversion of control), but they disagree on implementation. While Spring has become an empire, Guice claims to want to remain small and lightweight.

It is very interesting that the tendencies I’ve just discussed have been the subject of academic research. Recently, the Harvard Business School published a paper they entitled “The IKEA effect: When labor leads to love”, referring to the furniture retailer. They found that “When people construct products themselves, from bookshelves to Build-a-Bears, they come to overvalue their (often poorly made) creations”.

You can read this research yourself: Harvard Business Review summary of IKEA effect or ready the full IKEA Effect paper.

I’m not suggesting that frameworks are poorly made. On the contrary, some of them are very mature, show great insight and can save the developer many hours of work and greatly improve the end results of his efforts. What does concern me at times is the quasi-religious adoration some developers afford a particular framework. Before becoming a satellite to a framework, a developer should ask whether it’s a fit for the problem he has in front of him.

Much of the same reasoning applies to discussion of programming languages. Expertise in one language should not rule out other language possibilities if a new problem justifies a new approach.

Do I have to use a framework?

One might ask if it is necessary at all to use a framework. Martin Fowler has encouraged the use of plain old java objects (POJO) for business logic. Rod Johnson’s efforts have the objective of reducing coupling with a framework.

The purpose of the framework is to provide an abstraction for lower level libraries. This often includes scaffolding necessary to perform some common or frequent tasks. Everything that you can do in the framework can also be done directly in plain Java code, but the framework may save you time.

For example, Hibernate may make access to a datastore more object friendly. Spring may make integration more descriptive. GWT can both accelerate the development life cycle for rich web clients and at the same time reduce risk in deployment with Java compile time checks and cross browser issues taken care of.

The complexity of the problem I’m trying to solve will often determine whether or not I use a framework, and which framework might be a fit. If my application will have very little javascript then I probably don’t need GWT (or even jQuery). If there will be very few interface pages and session management is minimal then I probably don’t need Wicket. There are cases when jsp pages with simple includes are more than adequate and even more appropriate.

More on frameworks

I really liked this presentation I came across while looking into frameworks http://www.slideshare.net/mraible/comparing-flex-grails-gwt-seam-struts-2-and-wicket-presentation

Framework choice for WordPress plugin licensing

The focus of the current project is to create an easy to use RESTful API that will enable license generation and verification. That means there is very little Javascript and so GWT seems a poor fit. At this stage I’ve also eliminated Seam and Flex for Java.

Since Google App Engine for Java is the target for initial deployment, I did a review of many of the popular remaining popular frameworks available to see how well they play with GAE. In some cases, this might be a good time to reevaluate the choice of GAE and possibly lean toward Amazon EC2 if there were a compelling reason to choose a framework that didn’t perform as well on GAE.

The most interesting frameworks are wicket and Spring, but due to the low complexity and native support the first iteration of my application will use basic servlets and jsp.