Daniel Watrous on Software Engineering

A Collection of Software Problems and Solutions

Posts tagged gae

Software Engineering

Cloud hosting: Google App Engine vs. Amazon Web Services

My talented brother, Tim Watrous, has worked in the advertising industry for nearly 15 years now. His background started on the technical side, but he quickly found his strengths in the larger scope of marrying the business to a best possible advertising solution.

He and I often talk shop and tonight we got on the topic cloud computing. I mentioned some of my experience with the two most prominent platforms available, Google App Engine (GAE) and Amazon Web Services (AWS). The following details come from an email that I drafted for him to discuss these two platforms and some of the differences between them. Note that this is more high level than my previous discussion of GAE vs. AWS.

Google App Engine (GAE)

Google App Engine (GAE) is Google’s PaaS (Platform as a Service) offering. Unlike other mainstream hosting, they dictate the platform specifics and allow you to develop for it. This means it is not possible to run much off the shelf software on this platform. It is pay for use with a free quota, meaning small projects and testing are very cost effective.

Here’s a write up and video I did about GAE a long time ago:
http://freesourcing.org/blog/host-your-website-on-googles-servers-for-free

I also have a couple of running services that live on GAE, including this one (written in python):
http://freecontentspinner.appspot.com/

Recently I have published a lot on this site about using GAE for Java development.

Amazon Web Services (AWS)

Amazon Web Services (AWS) includes a growing suite of tools and technologies. Their main ‘hosting’ mechanism is EC2 (stands for Elastic Cloud Computing). This works a lot like a virtual server and can be setup in minutes. A principle difference between this and GAE is that AWS requires more consideration when deciding how the environment should be setup. There are times when that flexibility could both help and hurt a project.

AWS imposes fewer restrictions on what software you run and how you run it which makes more modern platforms accessible. Here’s an end to end example of an application that I developed and hosted on EC2 using Java and other modern technologies:
Java, Wicket and Hibernate on Amazon EC2 in a weekend

I have used the Amazon S3 (Simple Storage Service) for years now. It’s a type of content delivery network (CDN). I host files there that would consume too many resources on my main website and potentially make my site unresponsive. Most people never even know that I’m streaming video or downloading data from S3 instead of my own site because of how I define my URLs to amazon, like this video for my membership website:
http://media.wordpressmembershipuniversity.com.s3.amazonaws.com/video/wmu-optin-video-im.mp4

Most people look at the first bit and think it’s hosted directly on wordpressmembershipuniversity.com…

Misconceptions

I think the most common misconception most business folks have when they talk about cloud computing is the idea that it’s just a more robust or salable version of the solutions they have used in the past. In some respects that might be true. But there are some more significant differences.

Think about it like this. If you think about web hosting like a car, then you would be tempted to think about typical hosting like a little four cylinder economy car and cloud computing might seem like a V10 Dodge Viper. In reality, cloud computing is more like a build it yourself kit car and may be smaller than an economy car or larger than the biggest road car.

The importing point to be clear on is that it’s often NOT a drop in replacement for your current slow running web application.

Software Engineering

Wicket + GAE automatic reload

One disappointment of developing for Wicket and Google App Engine (GAE) is that the automatic monitoring and reloading of modified HTML files didn’t work. It had something to do with the single threaded nature of the GAE platform.

I had found a few previous efforts to make this work, but none of them worked with the current version of Wicket and GAE. I went without it for a while, but restarting the web server after every markup change finally drove me to figure it out.

Working with the project that I setup using my Wicket + GAE tutorial, I added two new files and modified the WicketApplication. Here are the details.

GaeModificationWatcher.java

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
package com.danielwatrous.softwarelicensing.web;
 
import java.util.HashSet;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.Map.Entry;
 
import org.apache.wicket.util.listener.IChangeListener;
import org.apache.wicket.util.time.Duration;
import org.apache.wicket.util.time.Time;
import org.apache.wicket.util.watch.IModifiable;
import org.apache.wicket.util.watch.IModificationWatcher;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
 
public class GaeModificationWatcher implements IModificationWatcher {
 
	private static final Logger LOG = (Logger) LoggerFactory
			.getLogger(GaeModificationWatcher.class);
 
	ConcurrentHashMap<IModifiable, Set<IChangeListener>> listenersMap = new ConcurrentHashMap<IModifiable, Set<IChangeListener>>();
	Duration pollFrequency;
	Time lastCheckTime;
	Object timeCheckLock = new Object();
 
	public boolean add(IModifiable modifiable, IChangeListener listener) {
		checkResources();
		HashSet<IChangeListener> listenerSet = new HashSet<IChangeListener>();
		Set<IChangeListener> listeners = listenersMap.putIfAbsent(modifiable,
				listenerSet);
		if (listeners != null) {
			return listeners.add(listener);
		} else
			return listenerSet.add(listener);
	}
 
	public IModifiable remove(IModifiable modifiable) {
		if (listenersMap.remove(modifiable) != null) {
			return modifiable;
		} else {
			return null;
		}
	}
 
	public void start(Duration pollFrequency) {
		LOG.debug("Starting watcher");
		synchronized (timeCheckLock) {
			lastCheckTime = Time.now();
			this.pollFrequency = pollFrequency;
		}
	}
 
	public void destroy() {
		// do nothing
	}
 
	public Set<IModifiable> getEntries() {
		return listenersMap.keySet();
	}
 
	public void checkResources() {
		Time now = Time.now();
 
		Time timeCheck;
		synchronized (timeCheckLock) {
			if (lastCheckTime == null) {
				return; // not started
			}
 
			Time nextTimeCheck = lastCheckTime.add(pollFrequency);
			if (nextTimeCheck.after(now)) {
				return; // nothing to do, not ready
			}
 
			// lets go
			timeCheck = this.lastCheckTime;
			this.lastCheckTime = now;
		}
 
		Set<Entry<IModifiable, Set<IChangeListener>>> entrySet = new HashSet<Entry<IModifiable, Set<IChangeListener>>>(
				listenersMap.entrySet());
 
		for (Entry<IModifiable, Set<IChangeListener>> entry : entrySet) {
			if (timeCheck.before(entry.getKey().lastModifiedTime())) {
				LOG.debug("Found modification, notifying listeners of change");
				for (IChangeListener listener : entry.getValue()) {
					listener.onChange();
				}
			}
		}
	}
}

GaeReloadRequestCycleListener.java

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
package com.danielwatrous.softwarelicensing.web;
 
import org.apache.wicket.Application;
import org.apache.wicket.RuntimeConfigurationType;
import org.apache.wicket.request.cycle.AbstractRequestCycleListener;
import org.apache.wicket.request.cycle.RequestCycle;
 
public class GaeReloadRequestCycleListener extends AbstractRequestCycleListener {
 
	public void onBeginRequest(RequestCycle cycle) {
		if (Application.get().getConfigurationType().equals(RuntimeConfigurationType.DEVELOPMENT)) {
			final GaeModificationWatcher resourceWatcher = (GaeModificationWatcher) Application.get()
					.getResourceSettings().getResourceWatcher(true);
			resourceWatcher.checkResources();
		}	
	}
}

WicketApplication.java

I haven’t provided all the details for this class, but this should show you how to implement it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
public class WicketApplication extends WebApplication
{    	
	...
 
	/**
	 * @see org.apache.wicket.Application#init()
	 */
	@Override
	public void init()
	{
		super.init();
 
		...
 
		// add your configuration here
		getRequestCycleListeners().add(new GaeReloadRequestCycleListener());
		IModificationWatcher watcher = new GaeModificationWatcher();
		watcher.start(Duration.ONE_SECOND);
		getResourceSettings().setResourceWatcher(watcher);
	}
}

Resources

http://agilewombat.blogspot.com/2010/01/wicket-on-google-app-engine.html
http://apache-wicket.1842946.n4.nabble.com/How-can-I-reload-HTML-in-app-engine-td3005241.html
http://apache-wicket.1842946.n4.nabble.com/Reload-html-in-Wicket-GAE-td4363236.html