On Pittsburgh and Progressive Enhancement

First off, Pittsburgh and Progressive Enhancement have nothing in common. However, I did just take a trip to Pittsburgh this past week and did some thinking on one of my favorite topics: Progressive Enhancement.

I’ve been thinking in recent months that progressive enhancement is an obsolete concept, at least in most circumstances and where it is not specifically dictated by requirements or regulations.

Certainly, JavaScript and Ajax are all the rage. Mobile devices are now as powerful as desktops with browsers equal to their desktop counterparts. Users expect rich functionality in web applications, which means JavaScript and DHTML. It is a waste of time and effort to build a non- JavaScript version of an application if 99% of your users will not see it. Supporting non JavaScript clients will also require considerable additional work.

But none of the above statements are true.

I took the trip Pittsburgh without a laptop, only a Blackberry Pearl and an iPod Touch. The Pearl is a bit old and always ran JavaScript questionably if at all and it only runs on a slow EDGE network. I therefore did not expect much from it.  However the Touch is an incarnation of the greatest device ever invented , the iPhone (please see the sarcasm here). I expected to be able to  do simply web-based tasks on the Touch, especially over its WiFi connection.

The only task I really needed to do that week was maintain my Ning site; essentially approve photos that are posted by users  and answering  messages from users if needed.

Unfortunately, neither device handled this simple task. While the devices share some of the blame, the fault is mostly Ning’s for not providing a simple non- JavaScript implementation of the needed functionality. Approving photos on Ning is as simple as opening the photos to approve page and clicking the “approve all” button (assuming you  want to approve all ). A pop up confirmation box is then displayed where you can confirm or cancel. When navigating to the page with the Blackberry, with JavaScript turned off by default, I was  informed  that JavaScript was required to view this page. After turning on JavaScript and clicking “approve all”, I was brought back to the top of the page with no apparent action or result. I reproduced this several times without successfully approving any photos. Of course, as I said before, I didn’t expect much from the Blackberry. That is why I had the iPod. However, it did the exact same thing and I was unable to approve the photos. I am not sure why the Touch didn’t work, but the functionality here is so basic, Ning could easily provide a static confirmation page for non JavaScript enabled browsers, and through progressive enhancement, enable the pop up for JavaScript enabled browsers.

While in Pittsburgh, I also tried to check my remaining  T-Mobile  minutes. I couldn’t connect to the T-Zones application ( a native app that comes installed) so I tried to access tmobile.com. I was curtly   informed the site didn’t support blackberry. This from the company that sold me the Blackberry and on the site where I can manage my Blackberry.

On a positive note, let me give kudos to Google’s excellent mobile Blackberry Apps. Gmail and Goggle Maps are extremely usable on the Pearl. I removed my native blackberry email account after accessing it through the Gmail mobile app.  Google Maps is also excellent, complete with traffic and satellite view and directions, but the T-Mobile connection speed made the experience a bit painful.

Right now, native applications are definitely the state of the art for mobile devices. Too bad they require custom development for each platform. I certainly imagine that a model driven approach than generates runtimes for multiple platforms may be more feasible in the future. The other option is that HTML sites, specifically HTML 5, allow browser based apps to provide a comparable experience and  supplant native apps. I would put my money in that category (actually I am). Maybe I am just too lazy, or too efficient, to want to rewrite apps for every platform I want to support.

I wrote a blog a while back about stumbleupon.com and how it didn’t support Oopera at the time and did not even let me proceed with that browser. I am not sure if it currently supports Opera (I have to think it does by now) but I still see sites that don’t support all browsers. HBO.com recommended to me only last night that I ditch Opera and I use IE or Mozilla. At least it gave me the option to proceed at my own risk. I took the risk and did not have a problem.

Progressive enhancement is still in my opinion the best way to develop web applications. The preferred approach is to build the app in three stages (for the initial release/revision). First build out the content layer (XHTML).  This step allows you to develop the necessary components that make up the core of the application, the guts of the M, the V, and the C. At this point, I usually am working with a raw HTML interface, perhaps with a bit of CSS just to make it usable. There may be some pages where the effort to do a non- JavaScript page is much more difficult and cumbersome to the user. In this case, It may make sense to require JavaScript for that specific module

Secondly, Style it (with CSS).  Add colors, positioning, etc.

Finally, and only where needed, add dynamic behavior with  JavaScript. I highly recommend jQuery for this.

The versions produced in each of these steps should be fully functional to the extent possible. This approach lets you focus on the basics of the application , foundation up, and not get caught up in what you think the behavior will be needed. Some of that may not be know early on  in the project.   Please understand this is only a very brief overview of my development process. I am not advocating a waterfall approach, only progressive development through progressive enhancement.

Looking back at BidBlazer

BidBlazer was developed and marketed by AD2001 Computer Services (which later became Infoblazer LLC) from 1999 through 2001. BidBlazer was one of the first in a new breed of online auction search tools and was the first client based tool in this niche. At it’s peak, BidBlazer searched eight online auction sites, including Ebay, Yahoo, Amazon, and MSN

During this time period, Ebay greatly consolidated it’s share of the online market. Therefore, other auction sites were left with few listings and most were eventually phases out. Due to the overwhelming market share of Ebay, cross auction searches became less useful. Developers moved to building tools targeted towareds Ebay specifically, such as sniping and submission utilities. In addition, development of online searching during that time period was very prone to changes in online auction formats. Since there was generally no published API or XML communication tools, such as later popularized by Amazon and Google, keeping up to date with auction site changes was a constant process.Therefore, BidBlazer was phased out in mid 2001.

For posterity, here are some screen captures from the software and a review of the product from Rocketdownload.com

Pigs Don’t Fly

A few posts back, I discussed my new project, an online quiz application, and the initial choice of GWT and GXT as the framework. The article was entitled “Pigs are Flying”, due to my previous convictions and statements that were not always kind to so called Rich Internet Applications.

Well it turns out after all that pigs do not fly and GWT/GXT does not fly.

A few caveats here. I use GWT daily in my consulting gig, so I have some experience in the area. I just remain a firm believer in applying Ajax judiciously. GWT (and certainly GXT) is full blown Ajax, almost HTML-less Ajax, if that is possible.

I still stand by the heuristic that an RIA will take twice as long to develop as the comparably functional Ajax-enhanced HTML application. Does this type of application have a place? Most definitely. Just be certain you requirements dictate such a tool and you have the funding and or time for its implement. In my situation, I just did not have such a luxury, particularly to wait around for applications to reload changes and to search out documentation on a less than fully documented third party framework.

Nuff said on that issue.

Since I’ve always been a user of Spring, and I initially started prototyping the quiz app in Spring MVC, I returned to SpringSource technologies, but not Spring but Grails.

I may have a little to say about Grails in the future (and I don’t like to repeat what is already said out there) but Grails is truly excellent.

While pigs are not flying anymore (although they are trying to take off, since I’ve also had some less than kind things to say about dynamic languages) there is still much to get used to in a language like Groovy. I specifically mean the lack of static type checking. However, IntelliJ does a decent job with things like code completion. Plus the dynamic application reloading is great. It’s nice not to hit the restart button after every code change.

That’s it for the status update. I’ll return to discussing performance issues in future articles. I find it hard not to get sidetracked when the topic of framework selection and evaluation arises.


HttpSession and GWT

Most anyone who has done Java server programming has dealt with the Session object, as well as its siblings (and I use that term in essence, not in fact) objects: ServletContext, HttpRequest and HttpResponse. Of course, if you are using a newfangled “Component” framework that tries to “shield” you from the intricacies of HTTP, you may not be familiar with these babies.

GWT is somewhat of a mélange of both Component frameworks and Request/Response frameworks. It is certainly on the component side of the spectrum, in that you are programming mainly in Java objects and using Swing-like constructs and events. But that is mainly on the client side of things. On the server side, GWT is strictly Servlet based, with some excellent marshalling thrown. The marshalling is so good that sometimes can make you forget the HTTP underpinnings. A dead giveaway, other than of course reading the documentation, is that any remote service must be defined in web.xml as a Servlet. I always found this odd, since most newer request /response frameworks(that gladly expose HTTP constructs) tend to use a single “Base” Servlet that you then must extend. Typically, you would create a single Servlet Mapping for this Base Servlet, which would handle essentially all requests. The Base Servlet, based on some xml, annotated, convention based, or a combination of the above, would pass control to a custom Java class which would handle the actual implementation. The Base Servlet and related framework will also provide much of the “plumbing” and other features, such as parameter object mapping, view integration, etc.

With GWT, every remote service you make available to the client must be defined as a distinct Servlet in web.xml (I feel like I’ve already said this, but it is worth repeating). The Servlet class defined is actually the developer’s specific java class, not a base Servlet. This class must extend RemoteServiceServlet, which is a descendent of HttpServlet. This is a key point in the discussion that will follow. Beware of a framework that forces you to extend its own classes (Ok, for the few of you in the know, XX Framework makes you do this as well, but I plead ignorance since I didn’t know better at the time. And I hadn’t seen Spring MVC yet).

Begin a Servlet, our GWT remote service implementation has access to all the HTTP goodies, specifically HttpSession, which I at one time said this post was about. To access the HTTP Session, you use the method


If this was a HttpServlet Sevice() method, you’d have access to this directly via the HttpServletRequest parameter . But since that method is long gone by the time your code is called, GWT stores this information and makes it available in a ThreadLocal object. ThreadLocal is a construct that lets each JVM thread have its own version of any class level variables. GWT needs ThreadLocal since a Servlet, as in HTTPServlet, needs to serve multiple threads. TheThreadLocal makes data available to all threads that may be accessing that Servlet.

Now remember the following point:

Each Servlet maintains its own ThreadLocal storage

There is nothing unusual with this. How else could multiple threads be served?

The problem arises when you decide to treat your Remote Service Implementation as just another Java class (POJO anyone) and not what it truly is, an HTTPServlet.

So the problem arises when any server side GWT code instantiates a GWT Remote service implementation directly.

Why is this a problem? Well in most cases, it may not be. We won’t include bad architecture in this discussion, which that most certainly is. The problem occurs when you try to access our old friend, getThreadLocalRequest().

The Remote Service Serlvet’s, which our Implemenation must extend, private ThreadLocal storage is initialized when the HTTP Service (POST, GET) happens. The HttpServletRequest, which includes the HttpSession is stored in the Servlet’s Threadlocal storage for access by the current thread.

When that Remote Service Implementation, or any other subsequent code, instantiates another Remote Service Implementation, the new Service’s ThreadLocal is created (it has to be as we are creating a new object) but it is not initialized, since no HTTP Get or Post has occurred. Therefore, any access to getThreadLocalRequest() will throw a NullPointerException, as this information has not yet been stored in ThreadLocalStorage.

I recently hit this problem while working with a client’s code. Doing an internet search and looking at some GWT books didn’t really shed any light on the problem. The only mention was a simple description, useful nonetheless, of the basic method for accessing the HttpSession and related objects. No warning or discussion of what happens when you instantiate a new Remote Service Servlet implementation object.

This is another reason why the SpringSource guys are being proved right time and time again. Please check out Spring MVC for an example of a web framework that can use pure POJO’s for most functionality. These POJO’s can be subsequently instantiated and used, as they have no dependency on the framework.

Implementing GWT, Pigs are Flying

I’ve always promoted a “standards” oriented approach to web development, meaning, a server side app that produces pure XHTML, styled with CSS, with Ajax added where appropriate. I’ve worked with GWT for some time and feel, while it is a tremendous product, it is essentially VB (ok maybe closer to Swing) for the web/JavaScript. It excels at building a desktop application look and feel. Contrast this with a more “web” look and feel, especially some of the sites associated with web 2.0.

I haven’t written a post for some time. I’ve been debating what client tool to use on quiz application. I originally planned on using Flex, figuring I could do a nifty interface and still integrate with Spring and Java on the back end. I also did some quick prototyping of CRUD screens in Spring MVC. It is amazing how easy it is to get simple HTML forms and data entry working  with that framework. I really like it and the philosophy behind it.

However, whenever I do HTML based apps, there is always the missing piece which is graphic design. I will admin that is not my strong point. I can do a passable job (see Domuswap.com archive), but it is not something that comes quick and easy to me. I would sure love to spend time learning photoshop, etc. I have no doubt that there are some nice approaches to easily styling HTML, I just haven’t stumbled across one that works for me yet.

So far, I can really get into Flex. I felt I was jury rigging the thing to get it to speak to Java/Spring. I wish Springsource would release an integration layer with GWT. I am sure it is coming. Not that is was that much work, but it is much more involved that what I’ve been doing for the past year with GWT. The client and server integration is so tight, you sometimes don’t realize on what side of the app you are.

The final blow came when I explored the excellent GWT add on library/framework, Ext GWT. The Web Desktop demo really blew me away. It is really just some nice CSS, but that is really important. The look of an application is the majority of it’s appeal.

I’ve always held that a major benefit of GWT is that it lets graphics impaired developers put together a good looking application. I think I may go down that path, or should I say, towards the dark side.

Always Start with the Use Cases

You usually come into any project with somewhat of an idea of what you want the thing to do. The first step, therefore, is to get these ideas down on paper. This process can encompass a mission statement, feature list, or other textual description of what you want to get done. We are in the brainstorming phase here, so anything you write is good. Let the ideas flow.

Now when you are ready to sit down and actually begin work (making sure your coffee cup is filled first) the first step should be use case diagramming. We are still in the brainstorming phase at this point, but use cases make the application begin to take shape, with early definitions of actual application components emerging.

Many of the use cases will be obvious. Some will become evident once the more obvious ones are defined. For use cases, there will certainly be “view available quizzes”, “take quiz” and of course some sort of authentication / log in behavior.

The initial data model can be developed in parallel with the use cases. Remember, both of these models will evolve throughout the life of the project. We are not doing big upfront design here, just enough architecture to get going. But some formal design is always a good idea.

As you go through the use case modeling, keep a note of the domain objects that you start describing. Start building your domain / class model now as well. Some of these will be obvious. In my cases, things like Quiz, Answer, Question, and of course the always present object User, were things I didn’t need to think much about. But the relationships between the two are not obvious. Is user related to quiz, quiz to question, or is there something more complex going on.

As soon as a few main uses cases are defined, it is time to jump into coding. Besides the fact that this is what I like to do, early coding will help you solve early problems and answer unknowns related to new technologies. You just want to make sure you are on the right track. This is called the vertical slice approach, where you build out a piece of functionality, generally a couple of use cases, from start to finish (from the web page to the database, the full stack). Plus it is always nice to have a working application to show. I won’t go into a full discussion of this topic. Just see any of the Agile literature for more information.

The quiz application is not yet an earth shattering innovation (but I do have a few things planned that could very well be). However, one goal is to implement the latest best practices in web application development. As such, I finally have the luxury to take the time and do it right. Of course, there are time pressures for this one as well, there always are. Even if there aren’t any dictated timelines, technology will eventually move ahead, you can’t stop that, and you will always be chasing the latest and greatest technology and practices.

The Complete Rewrite

Have you ever worked on a project and not thought that is would really benefit from a complete rewrite? It would be nice if that was not the case, but I believe reality dictates that what we wind up releasing is less than ideal.

One reason for this is changes in technology. New tools, techniques, and frameworks are constantly being developed. How many projects out there using Struts would not prefer a rewrite in a more modern framework.

The second reason is time constraints. We just do not always have the time to implement things the way we would like in the ideal world. There is always pressure to get things done yesterday. Much of programming wisdom is in finding the right balance between quick development and a well designed application, and delivering as much of both as possible.

We also seem to like the complete rewrite, such as when a new release of a product comes out. Check out this announcement from Spectral Core for their excellent Full Convert product.

Full Convert 5.0 released:

1-Conversion engine rewritten so that .NET framework is no longer required. This considerably reduces installation size and memory consumption, improves conversion speed and user interface responsiveness.

I feel good about this new feature, after all, it is a complete rewrite.

When I wrote Domuswap, I didn’t have the luxury of a complete rewrite. Domuswap was developed concurrently with the XX framework 1.1 and 2.0. Once the framework proved successful in Domuswap, it was released. However, I was still using the underlying framework and legacy code. Some was rewritten, but the time certainly wasn’t there for a complete rewrite. Indeed, I was very happy with how easily the existing code fit into the Spring MVC components that were added.

Would I have liked to do a complete rewrite? Definitely. I’d probably like to prove that XSL could indeed be speedy, or get rid of that approach entirely. I’d also like to enhance the multi threading techniques. I did this to some extent for version 2.0, in conjunction with my Sarasota Java Users Group presentation on concurrency. But that feature is still in experimental stages.

If I was to do a complete rewrite of the XX Framework, I would probably start with pure Spring MVC and just weave in some of the more interesting XX features, all the while sticking to their interfaces as much as possible.

All that being said, I find myself with the luxury of doing a complete rewrite at this very moment with the New Project. I plan to do this, in addition to satisfying the necessary functionality, as an implementation of what I see as the current best practices in web application development. I’ll have much more to say about this project in future columns.

The New Project

I began a new project today. We’ll really last weekend; I am just getting to write about it now.

The projects not in relation to my current full time consulting gig (which is a GTW project for the insurance industry). The new project will be an education testing program for a local community college, essentially online quizzes. I am also using this project as an opportunity to explore new tools and frameworks.

When we think education application, Flex comes to mind, and it is certainly a frontrunner, especially for the interface/testing component and if multimedia is incorporated.

For the back end, I am moving away from the XX Framework. While I think the framework is awesome, I need to get additional experience on more industry standard tools. Of course, XX is built on Spring MVC. But I want to get some additional experience in pure Spring apps.

Spring Web Flow is another possibility. The more I read about Spring MVC and Spring web flow, the more in common I see with what is in the XX Framework. Regarding Web Flow, I don’t know if I want to write so much in their XML/Expression based DSL. I don’t mind specifying this stuff for control reasons (like what XX does) but getting into defining variables doesn’t seem the place for XML based grammars.

Spring web flow does lend itself to a graphical editor. I haven’t tried Spring IDE, but since they were a recent SunJug sponsor, I took a quick spin in Skyway Builder , an eclipse based GUI for generating Spring MVC/Spring Web Flow apps. It is really an impressive product. However, being impressive doesn’t mean it is useful (sort of how I feel about GWT most of the time). Why try to graft a GUI over what is a perfectly capable and expressive language (Java / XML). In some cases, this will make sense, but in many or most, just writing plain old Java is quicker and quite understandable.

I need to work more with these tools to really make a judgment on them, however.

The one thing I always start with is a bit of upfront UML modeling. I use Enterprise Architect, which is a really inexpensive and reasonable good UML tool. Just brainstorm out the use cases and draw up some classed. It always helps to diagram these out before putting them to Java , and it doesn’t take too long to get to the Java stage.

One thing I did take away from reading some of the Web Flow docs was the use of UML state diagrams. I have always used Use Case diagrams and activity diagrams almost exclusively, but the web flow authors showed how State diagrams can more easily model the overall flow of the entire applications.

So I will most certainly do a Spring MVC / Hibernate / MySQLback end. Build out the model and services. I haven’t decided on the front end. It may be Flex, with BlazeDS or web service interface. Maybe JSF, which I have the least experience in and would like to learn. Most likely a portion will be in a Spring MVC view technology, namely JSF, Velocity, or Freemarker. Most likely, the application wills involve several of the above tools.

Java Versus PHP

One of my main interests is in web frameworks, or more simply put “what is the best way to implement a given application?”.

I like Java, love the IDE’s available and what they can do, and believe that a good Object Oriented design is the foundation of a good application, irrespective of the technologies used.

That being said, there are some really great PHP applications out there (WordPress, Joomla. Drupal, etc). You really can’t point to as many open source, easily available Java applications.

The main selling point of the PHP is the ease of installation and deployment. Most of the popular programs just require you to upload the source code and run an installation script.

Java on the other hand is a bit more involved. If all goes well, you should just be able to copy a War or Ear file to your server and you are all set. Of course, there are issues with restarting the application and such architecture is of questionable ease to extend with plug-ins the way PHP seems to be. I’d like to think that all the good Java developers are too busy building really cool corporate applications, which might actually be the case.

I bring all this up due to my recent experience in finalizing the XX Framework. I have two demo applications that are part of the distribution, a guestbook and an implementation of Java Pet Store. I’ve successfully converted these to XX 2.0, and they can easily be deployed as a War file.

My intention was to have the demos available online. This is always a nice to have when looking at a new framework or product.

I use and love Eapps.com for hosting. I’ve been able to run Domuswap.com on their shared VPS environment, but soon outgrew them and moved to a GoDaddy dedicated box.

Since I don’t want to spend much money to keep the demo apps online, I attempted to deploy the War to a shared 256MB server. It deployed and ran fine, but quickly blew through available CPU, so I had to back it out before I received a phone call from Eapps support “suggesting” that I upgrade.

I thought, if this was a PHP application, it would likely work fine and be quite fast. Is this just the nature of PHP versus Java, PHP being more suitable for the low end and Java more for the high end. I think this is probably the case. I should add, that the XX framework is a known CPU hog, being entirely dependent on XSL transformation and those can certainly spin up the processor.

I’d like to compare apples to apples some day and deploy a simple and lean Java app, with and without additional frameworks like Spring and Hibernate. I could rewrite the demos to do this and hopefully someday I will have the time to do just that.

XX Framework V2.0 is Released

After several years of work, the latest release of the XX Framework is now available.

The major change to the framework is that it is now ported to SpringMVC. This should allow greater flexibility and easier incorporation into existing or new Spring applications.

One of the first comments I heard when I released the framework back in 2006 was “how does this differ from Spring”. I didn’t think it was much like the Spring core, but he must have meant SpringMVC. Over the years, Spring MVC turned out to be one of the few frameworks that made perfect sense in almost every manner (as does Spring itself). Much of the early work on XX, and most frameworks, is in developing the plumbing (servlet routing, data marshalling, transactions, security, etc). This was all built into XX, but Spring already does it and much better I am sure.

A while back, I made the decision to port the framework to sit on top of SpringMVC and let it handle all the plumbing and let XX do what Spring does not: automatically marshall data from the web layer to the database layer and back, and provide an XSL centric view paradigm.

The migration was surpisingly smooth and would have happened much sooner if I can the time or additional resources. After the servlet mapping layer was migrated, most of the XX controller and database functionality just worked as is. The one main change I made was to incorporate Spring Hibernate integration (DAO stuff) and dependency injection.

The framework has powered Domuswap.com, in both its Spring and pre-Spring incarnations. While a site’s performance is certainly related to the hardward available, we did get some 4000 visitor days at its peak.

I still think the key differentiators of the XX framework continue to be valuable. Further integration into Spring is needed to where it is more of a plug in to Spring rather than a separate framework built on top of Spring (like Grails).

Check out the XX Framework.