Author Archive for christian

Come geek out with us this Thursday!

Thursday we’re opening the doors to the San Francisco office and hosting a casual gathering for our fellow geeks. Want to see what other developers are working on or who’s using the latest and greatest gem/practice/database/etc? Do you have a side project or open source framework that you’re contributing to (or just using) that you want to show off, lightning-talk style? Do you want to see what success others are having with agile development? Want to learn how to make a wicked macchiato on our Italian espresso machine? It’s all fair game!

We’ll have a room with a projector dedicated to lightning talks and the rest of the space will be free for mingling and socializing. There will be beer and food. Doors open at 5:30 and we’ll stick around until 7ish; if people are still hanging out we’ll migrate elsewhere for more drinks. Carbon Five is located at 171 2nd Street Floor 4 (map), above the Tempura House.

Lightning talk topics on the docket thusfar:

  • Alon will demo C5 Story Mapper
  • Corey will discuss http-pulse, a node.js application for monitoring http
  • Alex will discuss Wrong – a minimal assertion library for Ruby
  • Christian discusses the outcome of a recent MongoDB development spike
  • Jon discusses the Mission Artists United website
  • Jonah will discuss an iPhone widget he built over the weekend at the iphone deve camp

Please RSVP below if you’re coming.

See you Thursday!

We’re talking Agile at SFRuby…

Alon and I will be giving an hour talk about agile development next Wednesday at the SFRuby meetup:

Agile software development provides a number of discrete practices to help you be a better programmer and a great provider of programming services to your clients, internal or external to your business. The continuous daily rigor of testing, refactoring, pairing and integrating is a great path to producing quality software. The process of defining short development iterations with regular delivery and review with your client encourages collaboration and ensures that you are defining and meeting expectations along the way. We’ll describe the agile development and planning practices that we have found to be most effective in our work over the last 10 years and hope to hear from the group what is working for you.

The event is full, but there’s a waiting list… go ahead and sign up if you’d like to join us. Hope to see you there!

Updated! Slides…

Carbon Five is hiring!

We’re growing our team in both the San Francisco and Los Angeles offices (yay!). We’re looking for bright, passionate developers who love to learn and work with their peers. We’re writing a lot of Ruby and JavaScript these days, but there’s also a fair bit of Objective-C (iPhone) and some Java as well. We’ll cross-train awesome candidates in the technologies they don’t know. Are you an awesome Java developer who’s Rails-curious… perfect!

Check out our jobs page (http://www.carbonfive.com/jobs) for additional detail.

Intro to Spring MVC Talk at SD Forum

I’ll be speaking next Tuesday evening for SD Forum in Palo Atlo about Spring MVC. It’s mostly an introductory talk, but I’ll weave in some opinions and lessons learned from real projects. I’ll cover many of the MVC features, including those introduced in 3.0, by walking through code for a running application.

Event Page

Networking (and pizza) starts at 6:30 and we’ll get down to business at 7:00. The night starts with a short talk about Agile Java by Bjorn Freeman-Benson, of New Relic fame.

If you’re still doing Java and you’re curious about Spring MVC, come join in the fun.

Solid State Disks

I’m a bit of a hardware geek. I tend to keep up on what’s new and neat. I’m a bit conservative when it actually comes to buying the latest and greatest, but that doesn’t stop me from following the trends.

I’ve been eyeing solid state disks for the last year or so, speculating that they could have a huge impact on developer productivity once all of the major gotchas were worked out. I specifically though that pairing a decent laptop with an SSD would be awesome, since laptops generally have significantly slower hard disks than workstations. They’re also a bit slower across the board (cpu, memory bus, etc), so I thought the IO boost from an SSD might even things out.

In December I decided to end the speculation, so I bought an 80G Intel x25m (Gen 2) SSD for my 15″ Unibody MacBook Pro.
Continue reading ‘Solid State Disks’

Database Migrations: Fail when database is out of date

The latest release of the Carbon Five Database Migration maven plugin supports a new goal: check. The check goal fails the build if the database isn’t up to date. That is, if there are pending migrations the plugin produces a clear message explaining that the database is out of date and lists the pending migrations. Run mvn test and see something like this:

[INFO] ------------------------------------------------------------------------
[INFO] Building Gearlist - Data Access
[INFO]    task-segment: [test]
[INFO] ------------------------------------------------------------------------
[INFO] [db-migration:check {execution: default}]
[INFO] Checking jdbc:mysql://localhost/gearlist_test using migrations at src/main/db/migrations/.
[INFO] Loaded JDBC driver: com.mysql.jdbc.Driver
[WARNING] There are 2 pending migrations: 
 
    20100116010256_audit_tracking.sql
    20100121052539_add_daily_reports.sql
 
    Execute db-migration:migrate to apply pending migrations.
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] There are 2 pending migrations, migrate your db and try again.

It’s very easy to include the check goal in your build lifecycle if you’re already using the db-migration-maven-plugin.

...
<build>
    <plugins>
        <plugin>
            <groupId>com.carbonfive.db-support</groupId>
            <artifactId>db-migration-maven-plugin</artifactId>
            <version>0.9.9-m2</version>
            <executions>
                <execution>
                    <phase>validate</phase>
                    <goals>
                        <goal>check</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <url>jdbc:mysql://${db.host}/${db.name}</url>
                <username>${db.username}</username>
                <password>${db.password}</password>
            </configuration>
        </plugin>
    </plugins>
</build>

Check out the project home for additional documentation and details. There’s also a simple, complete example application showing off this configuration.

Enjoy!
Christian

C5 Test Support new addition: FunctionalTestRunner

We’re always looking for new ways to test our applications and we’ve been trying a few new things on our projects. One of the recent additions is a JUnit test runner designed to help make writing and running functional tests easier. In Javaland, we use Selenium and/or HtmlUnit for our functional tests. These are the tests that run against a deployed application over the wire using a real or simulated browser. Most of our functional tests work the application in the same way a real user would, testing sequences of realistic activity and often touching a number of pages. Since our functional tests use either a real browser or a simulated one, Javascript is executed and assertions made on the results. This gives us greater confidence that our app is really working, end to end.

Here’s the high-level flow that the functional test runner provides:

  1. Load fixture data from a DBUnit dataset.
  2. Download and install the application server (if necessary).
  3. Start the application server (using Cargo).
  4. Deploy the application, waiting until it’s completely started.
  5. Run one or more functional tests (using your preferred testing framework – Selenium, HTMLUnit, etc)…
    If a test dirties the database in a manner that must be reset, the test class can be marked with the @DirtiesDatabase annotation. This will reload the database fixture and optionally restart the application.
  6. Shutdown the application server.

Continue reading ‘C5 Test Support new addition: FunctionalTestRunner’

Web Application Testing @ SDForum October 6th

Updated: Added links to presentation and source code.

I’ll be down in Palo Alto speaking about Automated Web Application Testing Tuesday October 6th. If you’re interest in getting a peek at the typical Carbon Five Java web architecture along with a variety of strategies and techniques for testing, c’mon down and join us. The session will be primarily code-driven. I’ll implement a few features along with unit, integration and functional tests and show off some of the techniques and custom tools that help keep things simple and easy during development. Some of the topics include:

  • Brief overview of our typical web architecture and tools stack
  • Differences between Unit, Integration and Functional Tests
  • Dealing with the database (schema and fixtures)
  • Where does test coverage pay off the most?
  • In-browser and out-of-browser functional testing
  • Carbon Five best practices, custom tools and techniques
  • And more…

You can find the gritty details on the SDForum site.

Doors at 6:30 and the show begins at 7:00. Hope to see you there!

Here are the artifacts from the presentation:

Continuous Integration and Build Promotion

We have a build server and we practice continuous integration on all of our projects. In fact, it’s pretty much the first thing we set up after version control. We’re feedback junkies. It became especially apparent while working on a client project last year where we used their development infrastructure. They had a build server running, but the problem was that it took too long to get feedback. I was lost and jonesing. They had a monolithic build that took about 45 minutes to run its course and give me the affirmation I was seeking.

While setting up my current project, I decided to take a different approach when configuring our build server and continuous integration. I had a few goals in mind:

  • Super fast feedback
  • “Promoted” builds get pushed to the acceptance server
  • Minimize issues with the build infrastructure

I split our build into three plans:

Continuous Integration Build Plan

This resets the unit test database (using c5-db-migrations), compiles the project, runs all of the unit tests, and if there are no errors or failures, produces a war. The war is installed locally using Maven so that it’s accessible to other processes in a known location. This build is very fast and is triggered on every subversion commit. The command used for this build plan is:

mvn db-migration:reset clean install

Functional Tests Build Plan

This resets the functional test database, deploys the war built during the continuous integration build to tomcat using Cargo, runs all of the functional tests, and shuts down tomcat. This build is triggered on every successful continuous integration build (i.e. as a dependent build). A very short script performs the work of this build:

mvn -Pdev db-migration:reset
cd functional-tests; mvn -Pdev clean test-compile cargo:start surefire:test cargo:stop

And here’s the cargo-maven-plugin configuration:

<plugins>
...
    <plugin>
        <groupId>org.codehaus.cargo</groupId>
        <artifactId>cargo-maven2-plugin</artifactId>
        <version>1.0-beta-2</version>
        <configuration>
            <wait>false</wait>
            <configuration>
                <deployables>
                    <deployable>
                        <groupId>com.acme</groupId>
                        <artifactId>acme-web</artifactId>
                        <type>war</type>
                        <properties>
                            <context>acme-web</context>
                        </properties>
                    </deployable>
                </deployables>
            </configuration>
            <container>
                <containerId>tomcat6x</containerId>
                <zipUrlInstaller>
                    <url>http://www.apache.org/dist/tomcat/tomcat-6/v6.0.18/bin/apache-tomcat-6.0.18.zip</url>
                </zipUrlInstaller>
            </container>
        </configuration>
    </plugin>
</plugins>

Deploy to Acceptance Build Plan

This step doesn’t build anything or run any tests, but is a little more complicated than the others because it’s interacting with a another machine: our dedicated project acceptance server. We scp the war (built during the continuous integration phase) to our acceptance server. Next, we shutdown tomcat and clean a few things up (logs, old webapp, and work). Then we migrate the database (no reset because we care about the data). Last, we bring tomcat back up with the new war. This build is triggered on every successful functional test build. Here’s the script that’s run by the build server:

ARTIFACT_NAME=./ROOT.war
 
# SCP Application and Scripts
WAR=`find ~/.m2/repository/com/acme/acme-web -name "*-*.war" | sort | tail -1`
echo "Copying $WAR to acceptance server."
scp $WAR acme@acme-acceptance:$ARTIFACT_NAME
scp ./bin/*_as.sh acme@acme-acceptance:.
 
# Shutdown and Clean Tomcat
ssh acme@acme-acceptance sh ./shutdown_as.sh
 
# DB Migration
mvn -Pdev db-migration:migrate -Djdbc.host=acme-acceptance
 
# Install Application
ssh acme@acme-acceptance mv $ARTIFACT_NAME ./apache-tomcat/webapps/
 
# Startup
ssh acme@acme-acceptance sh ./startup_as.sh

Conclusion

This has worked out really well for the project and we get feedback very quickly.

You may wonder why we broke functional tests into their own plan. I find that functional tests can be a little less stable than unit tests (especially if selenium is involved), and they run much more slowly. I’ve seen cases where flaky functional tests caused a team to start ignoring build results because it was usually a problem with the infrastructure, not with the code. So, the decision was somewhat defensive and in retrospect, probably unnecessary.

We’ve spent less time maintaining our build plans than in the past as well. At some point, our build server MySQL instance crapped out and even though all of the databases were deleted, our builds all ran successfully when the build server came back up because they start with a database reset, which creates the target database and migrates it to the latest schema.

A single war is promoted as it passes a greater level of testing, and is eventually deployed to the acceptance server if all of the tests pass. While we’re saving a little time by not rebuilding the archive for each plan, that’s not the only thing I like about it. It just feels a little more right and it completely eliminates the chance that something about the artifact changes as it makes its way through the pipelines. The same war can make its way from the first CI build all the way to production. This is possible because we include default configuration for the application which matches our development environments, and then provide a mechanism for externalizing application configuration for the one-off environments.

I think most modern build server software provides everything you need to do something like this as it’s rather straight forward. However, for those who are curious, we have been using Bamboo for the last year or two and recently installed TeamCity so that we can give it a proper try. Both are great products, and if you’re on a smallish team, TeamCity is completely free (and is superior to the open source alternatives, IMHO).

How are you using your build server?

Stripes: A Successful First Project

We’re wrapping up a project that I’ve been leading since September and I’ve been reflecting on some of my decisions. Some of this reflection might be interesting to other developers. There are a few things on my mind, but I’ll start off off with my decision to use Stripes as our MVC instead of our usual, Spring MVC.

Background

I’ve never been completely satisfied with Spring MVC (note that it’s pretty hard to win me over completely). We know it well and we’ve had many successful projects while using it. We’ve also used many of the new features that came along with Spring 2.5 (@Controller, more convention over configuration, etc), but in the end I still wasn’t loving it.

I came across Stripes over a year ago, and noted that it had a small but fairly vibrant and excited community. The projects goals definitely resonated with me:

  • Make developing web applications in Java easy
  • Provide simple yet powerful solutions to common problems
  • Make the Stripes ramp up time for a new developer less than 30 minutes
  • Make it really easy to extend Stripes, without making you configure every last thing

So I decided to give it a try on a real project. Switching from something we know inside and out to something that none of us had production experience with was arguably risky, so we decided to give it a try for a week with the intention that we’d go back to Spring MVC if anything took too long or felt awkward. Thankfully, that never happened.

Some of this article may read a bit like a Stripes versus Spring MVC comparison. That’s not really my intention, but it’s somewhat inevitable as much of my experience has been with Spring MVC. This isn’t intended to be a Stripes tutorial (there are great ones out there), so the code snippets and technical details will be sparse.

Controller Lifecycle, Binding, and the Model

Stripes controllers are called “Actions” or “ActionBeans” and each incoming HTTP request is routed to one primary Action (like Spring MVC). Stripes creates a new Action instance for each incoming request; Spring Controllers are singletons in comparison. Stripes binds parameters into the fields on the Action where Spring MVC passes them as method parameters. The Stripes Action is not only the “Controller” in the MVC, but it also serves as the root of the “Model” as well. The Action is made available to the View and all properties with getters can be queried using JSP-EL. Spring’s model is separate, necessitated by the Singleton nature of the Controller.

Let’s look at a simple example:

// URI and embedded parameters defined using CleanURLs
@UrlBinding("/status/{orderId}/{$event}")
public class OrderStatusAction extends AbstractActionBean
{
    // Spring managed service to be dependency injected (see "Worth Mentioning" below)
    @Autowired OrderService orderService;
 
    // Required incoming parameter bound in the URI with {orderId}
    @Validate(required = true, minvalue = 1) long orderId;
 
    // OrderStatus to be accessible from the view for rendering
    OrderStatus orderStatus;
 
    // public setter tells stripes to allow binding
    public void setOrderId(long orderId) { this.orderId = orderId; }
 
    // public getter tell stripes to allow access from the view
    public OrderStatus getOrderStatus() { return orderStatus; }
 
    public Resolution view()
    {
        orderStatus = orderService.getOrderStatus(orderId);
        if (orderStatus == null) return new ErrorResolution(404);
        return new ForwardResolution("/WEB-INF/jsp/order-status.jsp");
    }
}

In our view we can access the Action / Model:

...
<jsp:useBean id="actionBean" scope="request" type="eg.OrderStatusBean"/>
...
<li>Order Number: ${actionBean.orderStatus.order.id}</li>
<li>Status: ${actionBean.orderStatus.status}</li>
<li>Tracking Number: ${actionBean.orderStatus.trackingNumber}</li>
...

This example binds to a long, which is pretty simple. Stripes can bind into graphs of objects, instantiating them along the way if necessary. Collections are fully supported as well. We haven’t yet found an example of something we couldn’t bind into right out of the box.

While I first was resistant to the Stripes lifecycle and combination of Controller and Model, I soon warmed up to it and now I find it quite natural, a bit better from a code readability standpoint, and more aesthetic. It’s just the right amount of abstraction and encapsulation to make for speedy development while being easy to maintain. I really like the fact that new instances of actions are created for each request, because the alternative is to pass all of your state into a handler method, which can easily lead hard-to-read code, especially with the annotations required to describe which request parameter maps to which method parameter.

If you can’t bind directly into your value objects and entities, the Action gives you a great place to bind into first, allowing you to manually instantiate your domain objects plugging in values from the Action. This is quite useful when your domain model isn’t direct-binding friendly because of invariant enforcing, immutable value objects, and other practices encouraged by Domain Driven Design.

We keep our Actions simple and lightweight, deferring all non-display logic to transactional, spring-managed services. I’ve seen examples where Actions are directly interacting with the database, a pattern I discourage.

If you want to know more about how Stripes works, check out the references section at the end of this article.

Generating URIs in Views

I can’t tell you how many times I’ve run into regressions after making changes to URIs where a page would link to a controller at the wrong URI. With Stripes, your URIs are defined once and only once, so when you change where an Action lives, pages will link to it correctly at its new location. Stripes tags take a beanclass argument so that it can determine the correct URI at runtime rather than hard-coding it in the view.

<stripes:link beanclass="eg.OrderStatusAction" event="view">
  <stripes:param name="orderId" value="65432"/>
  View Order Status
</stripes:link>

Renders: <a href="http://example.com/status/65432/view">View Order Status</a>

The <stripes:url …/> and <stripes:form …/> tags work the same way. To round it out, Actions can forward or redirect to other Actions without embedding URIs:

return new ForwardResolution(OrderHistoryAction.class, "view").
return new RedirectResolution(OrderHistoryAction.class, "view").

The net result is that there’s a single definition of each URI in our system and it lives on the Action which handles that URI, realizing the Don’t Repeat Yourself (DRY) principle. We’ve been able able to change our URIs easily without fear of breaking views, which has been helpful as the project grows.

Configuration

Stripes only needs a few lines of configuration in your web.xml. That’s it (really). Stripes was built with convention over configuration in mind from day one. Actions and Extensions (Converters, Formatters, Interceptors, etc) are auto-discovered via classpath scanning. We never found ourselves needing to configure something differently than how it was out of the box.

While you can configure Spring MVC to be convention based (it’s okay to chuckle at this too), it’s not that way out of the box. Perhaps Spring 3.0 will change this, but I have the sense that no matter what, there will always be some evidence of the fact that Spring MVC’s internals allow a wide range of configuration.

Converters and Formatters

In other frameworks, these concepts are often conflated into a single class. Stripes converters do one thing: convert from strings to objects (e.g. phone numbers, zip codes, etc). Whenever you need to turn an incoming request parameters into something more than a string, the converter is there to help.

Formatters work the other way, formatting objects into something that looks right on the screen as text. Formatters can support multiple format types, so that you can support displaying objects differently when necessary (e.g. phone with extension, zip code with a plus 4, etc).

These helpers are simple to write and test. All of the stripes tags will use them if they’re present, so it’s easy to affect how something — phone numbers for example — are displayed across the entire application.

Testing

There’s no reference to the servlet API in your Action classes, so it’s easy to write tests against the Java code within. Tests fall into two categories: very lightweight unit tests which only test what’s happening in your handler and slightly more heavy-weight tests which involve more of the stack (but not the servlet container).

The outline for a unit test goes something like this:

  1. Instantiate your Action
  2. Inject service stubs/mocks
  3. Use the public setters to specify values to necessary fields
  4. Invoke the handler
  5. Assert on the Resolution and the state of the Action (optionally your Stub)

Note that all of our Action tests fall into this category, even though you can test more of the stack (URL binding and validation).

What you can’t do — I haven’t seen any Java MVC provide this though — is write tests against the rendered markup of your views (a la Rails) without bringing up the servlet container.

Read more about testing with Stripes here.

Documentation and Community

The Stripes documentation is definitely not very complete or polished. Some of the documentation is out of date or non-existent (e.g. CleanURLs). The same goes for the examples. There is a decent book from Pragmatic Programmers press however, which I recommend if you’re interested in Stripes.

The good news is that despite all of this, it’s easy enough to find or figure out what you need without too much fuss. Piecing together examples, tutorials, bogs, documentation, etc ultimately gives you what you need. The Stripes source is small enough that you can rummage through to see how things work easily. It’s not as configurable as Spring MVC so the code is less abstract and a little easier to grok (though it’s not as elegant).

There is an active mailing list where the developers and other users help out with questions.

Conclusion

I think working with Stripes is a lot of fun and that we made the right decision to use it. I’d go as far as saying that we were at least as productive with it as we would have been with Spring MVC, and it’s likely we were more productive. I’d say that the authors have largely delivered on their goals. One of our front-end developers quickly dove right into building Actions without much help from other developers. While we had the occasional “Huh?!” moment while trying to figure out why something wasn’t working, they were few and far between.

In comparing it to Spring MVC, I think there’s a simplicity and elegance to Stripes that comes from it being just an MVC and it not having the same legacy as Spring. While Spring MVC has certainly evolved, it’s brought some of its crufty parts along with it. I’ll be keeping an eye on Spring MVC to see what’s in store with 3.0, and I hope to be proven wrong.

There’s no doubt that Stripes is in a niche as compared to many of the other web frameworks. The community is much smaller, and the development cycle much longer (last release was August 2008 and the one before that May 2007), which sometimes makes me wonder what Stripes future holds.

I’d say that one downside to Stripes, or any other framework that has a single backing Action per URI, is that there isn’t a great story for dealing with pages that aggregate a number of features and those features also show up on other pages. The problem is that we can’t rely on the Action to provide all of the reference data, so we have to rely on other mechanisms for fetching it (filters, interceptors, tags, etc). It’s not an issue with Stripes specifically, but all framework with the same approach.

For projects where this single primary controller per URI limitation isn’t a problem, I would definitely use Stripes again and I think it’s a framework Java developers should look into if they aren’t completely happy with whatever they’re using.

Also Worth Mentioning…

There are lots of other neat features in Stripes too: validation annotations and helper methods, stripes layout, flash scope, wizard forms, encrypted parameters, a JavaScriptResolution for serializing Java objects to JavaScript, etc. Open up the Stripes jar and start looking around.

Our application is using Spring as an IoC container for everything behind our Actions; to get handles to your Spring managed services we use a simple Stripes Interceptor which injects dependencies into Actions.

When using CleanURLs, you’ll want to use the DynamicMappingFilter, though there’s not much mention of it in the documentation. CleanURLs in Stripes 1.5.1 should be even more flexible (see STS-617).

We used Spring Security on our application found that Stripes and Spring Security play nicely together.

Spring 3.0 will include a few new features that are similar to features I really like in Stripes, including RESTy URLs, and tags for generating the URIs to controllers. The downside is that it’s still using the singleton model, which equates to controller handler methods with potentially lots of annotated parameters.

References

Updates: added reference to recently published Spring 3.0 MVC blog.