We’ve got a good range of perspective and experience to draw from. Maria Giudice from Hot Studio is our user-centered designer, Rob Spiro from Aardvark is our user-centered product entrepreneur, Darren David from Stimulant is our multi-touch and NUI innovator, and I am our Agile software developer.

This event seems to have really caught on to the growing energy in the entrepreneurial community around strategies and techniques for creating products that people really want. Eric Ries is giving the keynote on Tuesday just one year after his first big conference presentation at Web 2.0 2009 and Steve Blank is giving the keynote Thursday. Both of these guys and the growing community around them have been inspiration to me and our work at Carbon Five over the last year for the answers they provide about how software development serves the goal of creating successful businesses and products.

We use Pivotal Tracker to manage our feature backlog and the day-to-day activities of tracking a feature from estimation through delivery and acceptance. However, we find it easy to lose the big picture in the detail-oriented view Tracker provides.

Story Mapper uses your project data from Tracker to provide a higher level view geared toward release and milestone planning. This view is based on Jeff Patton’s story mapping techniques and provides the ability to see how the different components of your project will evolve across successive milestone.

All you need in order to use Story Mapper is a Tracker project with a backlog of stories and a subset of labels that represent user activities or components of your system. If you have a large project and have not been using labels in this way, you may have some work to do in Story Mapper before you have a story map that makes sense.

Try it out, let us know what you think, and tell us how to make it better. Comment here so others can see your thoughts or by contacting us directly.

It’s not in every agile software team’s bag of tricks. Asking around our fairy savvy office, I discovered it’s far from universal. In the “State of Agile” report from Version One, which includes survey results from 2500 software developers, it wasn’t mentioned. Since I haven’t seen it show up that much in other agile writings, I thought I’d share my experiences here.

What is “5 Whys”? I picked up “5 Whys” from the lean software movement, which sprang from Toyota manufacturing. You can read about its history on wikipedia, but it’s pretty simple: at the end of the assembly line, when a widget comes out with a problem, you stop the line and ask “Why?” Whatever the reason, you ask the “Why?” again. Repeat at least 5 times. The goal is to discover the “root cause” of the defect, and fix the root cause, not just some symptom. Wikipedia has a good example around car repair. Here’s a software example:

1. Why did Sheryl say the sign-up flow was broken?
A: Because she was trying to sign up a second time. Duh.
2. Yeah, but why did the flow break if the user is already signed up?
A: It’s a bug. We didn’t have a test case for that.
3. Why didn’t we have a test case for it?
A: I just didn’t think of it.
4. Why didn’t you think about it?
A: The story seemed easy– I guess I didn’t think it through.
5. Why do you think you didn’t think it through?”
A: I guess I was working alone and was in a hurry.
…

Obviously you don’t have ask “Why?” exactly five times, but that does seem to be a pretty good number. The point is you start articulating the behaviors and procedures that cause the problems.

Taking It for a Spin. I’ve come up with exercise for agile software teams. Like I mentioned, when quality drops below your comfort level, give it a try:

1. (2 minutes) Make a list of the most recent bugs you’ve uncovered. I’ve found that the last 10 is usually enough– if not too much. You can put these on a white board, index cards or a wiki page– all will work.
2. (2 minutes) Prioritize them based on the ones you want to talk about. Which caused the most embarrassment or cost? You usually only have energy to talk about 5 or 6 of them.
3. (15 minutes) For each bug, ask “Why did this happen?” Then, probe deeper until you can’t get much further. Write the answers down where everyone can see.
4. (5 minutes) When you’re done with all the bugs (or out of time), circle common root causes.
5. (10 minutes) Brainstorm ways to mitigate or eliminate the root causes. Come of with one or two SMART goals for the team.

Some of the root causes we’ve gotten to recently:

• we didn’t pair on hard features
• we didn’t ask for clarifications on the requirements
• one engineer didn’t understand the intent behind the code
• nobody looked at the app on Internet Explorer 6

Just like in agile reflections, the changes made coming out of these meetings stick with the team. During one session, the developers decided that there were just too many mistakes that someone else would have easily caught– so they always needed two sets of eyes on every checkin. I’d advocated this before, but it was never applied consistently. To my surprise though, after the reflection, everyone was  disciplined about this policy– and our quality was much better.

The Problem

On our work on gobalto.com, we spend time to have good fixture data for our tests– data that can represent all the important application states that our tests require. As a result, our tests are very dependent on the data. It’s important that someone doesn’t inadvertantly change it in subtle ways. This has led us to write not only asserts at the end of tests, but pre-conditions as well. For example,

    ...
    inotech = companies(:inotech)
    assert inotech.services.public.include?(categories(:a))
    assert inotech.services.public.include?(categories(:b))
    assert inotech.services.public.include?(categories(:c))
 
    post :edit_services_dialog, :id=>inotech.id,
                                          :service_category_id=>categories(:a).id
    inotech.reload
 
    assert inotech.services.public.include?(categories(:a))
    assert !inotech.services.public.include?(categories(:b))
    assert !inotech.services.public.include?(categories(:c))

Although the pre-conditions were introduced to guard against accidentally changing fixture data (or just to figure out what’s going on), we don’t necessarily delete them. They provide stronger tests. And in some ways, leaving in pre-conditions make the code more readable by providing documentation of your assumptions to the readers. The code above would be hard to follow without the clarifying pre-conditions. What do we expect to change and what stays the same?

Unfortunately all these asserts make the tests twice as long. And there’s a subtle readability problem: there’s no relationship between the corresponding pre- and post-conditions. In the example above, you have to scan carefully to see that the three assertions are repeated, but negated (b and c only). The reader must mentally put pieces together. And it’s not DRY. Using local variables doesn’t help much.

The Solution

I was inspired by a nice little test helper called assert_difference. It takes a string to evaluate and a block to execute. It’s useful for checking on state changes– especially database changes– during a test:

assert_difference('Company.count', -1) do
    Company.delete_one
end

(Without this method, we rely on a count of all database records, or merely look for specific ones. The former approach leads to brittle tests, and the latter to incomplete assertions.)

A limitation of assert_difference is that it only deals with integers. What if it were generalized? Here goes:

    i = true
    assert_changes 'i' => false do   # read as: i changes to false
      i = false
    end

The string passed to assert changes is evaluated in the block context, both before and after the block is run. So this block asserts that i becomes false (and by deduction, starts out as true). It executes asserts on both ends, just like we want.

Of course sometimes you want to be explicit about a state change, so you can specify both the starting and ending values using an array:

    o.answer = 'yes'
    assert_changes 'o.answer' => ['yes','no'] do
      o.answer = 'no'
    end

To handle the original example, you can pass multiple pre/post conditions of arbitrary complexity and they are all evaluated before and after the block is executed:

      assert_changes 'post(:a).status' => [:preview, :published],
                            'comment(:c).status' => [:preview, :deleted] do
        ...
      end

Finally, I added support for a :no_change symbol. Now I can re-write my original problem in a clearer form:

    assert_changes
        'inotech.services.public.include?(categories(:a))' => [true, :no_change],
            'inotech.services.public.include?(categories(:b))'=>false,
            'inotech.services.public.include?(categories(:c))'=>false do
      post :edit_services_dialog, :id=>inotech.id, :service_category_id=>categories(:a).id
      inotech.reload
    end

For completeness, I added assert_no_changes, with slightly extended parameter possibilities:

    i,j = 'hello','hi'
    assert_no_changes 'i' do ...  # i (before) == i (after)
    assert_no_changes 'i'=>'hello' do ... # i == 'hello' before and after
    assert_no_changes ['i','j'] do ... # neither i or j change
    assert_no_changes 'i'=>'hello','j'=>'hi' do # or be explicit

Update: This code and documentation is now available on github: http://github.com/ndp/assert_changes/tree/master

Thoughts? Comments?

On those occasions, I supplement my list in Tracker with another technique for capturing user stories. It involves modeling using a UML IDE. Primarily, I use Activity Diagrams because they capture behavior using ‘actions’ and allow for flow. Because activity diagrams are essentially flow charts, they provide a simple comprehension mechanism for all customers regardless of their technical background. I usually display my desktop in the conference room using a projector and interactively add elements to the diagram as I receive customer input. For demonstrating modeling techniques, I will use a ‘Music Store’ web application as an example.

Summary of Steps:

1. Create an Activity Diagram from a High-Level Functional Requirement.
2. Capture Customer Input for Identified Functional Requirement.
3. Model Flow for Activity, Link Actions with Diagram Elements.
4. Aggregate Similar Actions to Separate Activity.
5. Convert Activity Diagram into User Stories.
6. Repeat Steps until all Functional Requirements are Exhausted.

Step 1: Create an Activity Diagram from a High-Level Functional Requirement

Start with a known high-level functional requirement and create an Activity diagram. When starting, I will identify a large chunk of functionality that I know to be required for the project and use that as a discussion point with the audience. For my sample Music Store project, I would choose ‘Browsing the Inventory’ as a large piece of functionality. At this point, the audience might begin to shout out other large pieces of functionality (i.e. ‘Customers can create accounts’, ‘Employees can add to inventory’). If this occurs, I like to create empty Activity diagrams named appropriately. These large pieces of functionality will eventually be decomposed into smaller, more manageable, Activities and Actions.

High-Level Functional Requirement as an Activity

Step 2: Capture Customer Input for Identified Functional Requirement

Gather input from the audience regarding this Activity / functional requirement and capture pertinent information as Actions. Essentially, what you are trying to do here is decompose the larger ‘Browsing the Inventory’ functional requirement into smaller ones. Make an attempt to name each of the Actions with story-like syntax (i.e. ‘As a <role> I can <result> so that <benefit>’). The goal of this step is to create a catalog of smaller functional requirements so that you can start modeling flow.

Capturing Smaller Functional Requirements as Actions

Step 3: Model Flow for Activity, Link Actions with Diagram Elements

Start modeling flow for this Activity, linking Actions with other diagram elements. All Activities must have a start as well as an end point. I model my Activities so that my flow is complete. This ensures that all known paths have been captured. This step is crucial in capturing all known functionality for the high-level requirement.

Remember that this is an interactive step with the audience. The audience / customer will provide input as to the flow of the Activity from Action to Action.

Adding Flow to Activity

Step 4: Aggregate Similar Actions to Separate Activity

If the diagram begins to flow outside of your viewable area, this is an indicator for breaking out one or more actions into a separate Activity. Several actions can be converted into one abstract action that encompasses all the functionality provided by all those actions. In the figure in step 3, the Action ‘Purchase an Item’, was created to represent another Activity diagram. The separate Activity diagram that I created was named ‘Purchasing an Item’ and contains the detailed flow for that functionality.

This step is reliant on the Tech Lead’s skill to identify similar areas of responsibility within the activity. 

Step 5: Convert Activity Diagram into User Stories

After one Activity diagram is fairly complete, begin entering them into Tracker as Stories. Each Action should be converted into a user story using a consistent syntax (i.e. ‘As a <role> I can <result> so that <benefit>’).

Convert Action elements into User Stories

User Story list with Activities as Milestones

 

Step 6: Repeat Steps until all Functional Requirements are Exhausted

Steps 1-5 must be repeated until all high-level functional requirements have been modeled as Activities during discussion. In the end, I will have a catalog of Activity diagrams. In the figure below, a Package Diagram displays all the Activity Diagrams that I have modeled. I like to use each of the Activities as a milestone during planning. This may or may not be ideal depending on how granular your functional requirements are.

Package Diagram: Inventory of Activities as Milestones

 

Conclusion

There are many arguments against modeling in the agile communities, so it is important to view modeling as being supplemental to the process. In the end, our models are used to aid in the process of conveying and gathering information with the customer. Some other benefits include:

• The diagrams can serve as a documentation deliverable
• Interactive modeling along with the diagrams, which serve as a visual aid, improves customer comprehension of our software development process
• This is a great way to capture requirements when none exist
• The Activity diagram allows us to write stories in a format that follows an accompanied wireframe (if available)
• User roles can be added to Activity diagrams as swimlanes (partitions). This aids in separating functionality by use case

While setting up my current project, I decided to take a different approach when configuring our build server and continuous integration. I had a few goals in mind:

• Super fast feedback
• “Promoted” builds get pushed to the acceptance server
• Minimize issues with the build infrastructure

I split our build into three plans:

Continuous Integration Build Plan

This resets the unit test database (using c5-db-migrations), compiles the project, runs all of the unit tests, and if there are no errors or failures, produces a war. The war is installed locally using Maven so that it’s accessible to other processes in a known location. This build is very fast and is triggered on every subversion commit. The command used for this build plan is:

mvn db-migration:reset clean install

Functional Tests Build Plan

This resets the functional test database, deploys the war built during the continuous integration build to tomcat using Cargo, runs all of the functional tests, and shuts down tomcat. This build is triggered on every successful continuous integration build (i.e. as a dependent build). A very short script performs the work of this build:

mvn -Pdev db-migration:reset
cd functional-tests; mvn -Pdev clean test-compile cargo:start surefire:test cargo:stop

And here’s the cargo-maven-plugin configuration:

<plugins>
...
    <plugin>
        <groupId>org.codehaus.cargo</groupId>
        <artifactId>cargo-maven2-plugin</artifactId>
        <version>1.0-beta-2</version>
        <configuration>
            <wait>false</wait>
            <configuration>
                <deployables>
                    <deployable>
                        <groupId>com.acme</groupId>
                        <artifactId>acme-web</artifactId>
                        <type>war</type>
                        <properties>
                            <context>acme-web</context>
                        </properties>
                    </deployable>
                </deployables>
            </configuration>
            <container>
                <containerId>tomcat6x</containerId>
                <zipUrlInstaller>
                    <url>http://www.apache.org/dist/tomcat/tomcat-6/v6.0.18/bin/apache-tomcat-6.0.18.zip</url>
                </zipUrlInstaller>
            </container>
        </configuration>
    </plugin>
</plugins>

Deploy to Acceptance Build Plan

This step doesn’t build anything or run any tests, but is a little more complicated than the others because it’s interacting with a another machine: our dedicated project acceptance server. We scp the war (built during the continuous integration phase) to our acceptance server. Next, we shutdown tomcat and clean a few things up (logs, old webapp, and work). Then we migrate the database (no reset because we care about the data). Last, we bring tomcat back up with the new war. This build is triggered on every successful functional test build. Here’s the script that’s run by the build server:

ARTIFACT_NAME=./ROOT.war
 
# SCP Application and Scripts
WAR=`find ~/.m2/repository/com/acme/acme-web -name "*-*.war" | sort | tail -1`
echo "Copying $WAR to acceptance server."
scp $WAR acme@acme-acceptance:$ARTIFACT_NAME
scp ./bin/*_as.sh acme@acme-acceptance:.
 
# Shutdown and Clean Tomcat
ssh acme@acme-acceptance sh ./shutdown_as.sh
 
# DB Migration
mvn -Pdev db-migration:migrate -Djdbc.host=acme-acceptance
 
# Install Application
ssh acme@acme-acceptance mv $ARTIFACT_NAME ./apache-tomcat/webapps/
 
# Startup
ssh acme@acme-acceptance sh ./startup_as.sh

Conclusion

This has worked out really well for the project and we get feedback very quickly.

You may wonder why we broke functional tests into their own plan. I find that functional tests can be a little less stable than unit tests (especially if selenium is involved), and they run much more slowly. I’ve seen cases where flaky functional tests caused a team to start ignoring build results because it was usually a problem with the infrastructure, not with the code. So, the decision was somewhat defensive and in retrospect, probably unnecessary.

We’ve spent less time maintaining our build plans than in the past as well. At some point, our build server MySQL instance crapped out and even though all of the databases were deleted, our builds all ran successfully when the build server came back up because they start with a database reset, which creates the target database and migrates it to the latest schema.

A single war is promoted as it passes a greater level of testing, and is eventually deployed to the acceptance server if all of the tests pass. While we’re saving a little time by not rebuilding the archive for each plan, that’s not the only thing I like about it. It just feels a little more right and it completely eliminates the chance that something about the artifact changes as it makes its way through the pipelines. The same war can make its way from the first CI build all the way to production. This is possible because we include default configuration for the application which matches our development environments, and then provide a mechanism for externalizing application configuration for the one-off environments.

I think most modern build server software provides everything you need to do something like this as it’s rather straight forward. However, for those who are curious, we have been using Bamboo for the last year or two and recently installed TeamCity so that we can give it a proper try. Both are great products, and if you’re on a smallish team, TeamCity is completely free (and is superior to the open source alternatives, IMHO).

How are you using your build server?

The post is the first of a series on user experience (UX) design practices for agile software development teams. In an upcoming project with our partner Hot Studios we will be exploring new ways to integrate their user experience design process with our software development process. We’ll share some learnings from that, experiences from the past and new ideas for the future.

First we have a growing reading list of existing resources on the topic that I am posting here for you. If you have other resources for us, please share.

User Research as a Commodity
Added 2008-02-14
Alex at Pivotal Labs noted this post from a user experience manager at Kaiser Permanente who describes a strategy for including regular user testing in an agile development process. The strategy is attractive because it fits with a key characteristic of agile where simple practices performed regularly produce high level complex results.

Kent Beck v. Alan Cooper
This is a 2002 article with the following abstract “Kent Beck is known as the father of ‘extreme programming,’ a process created to help developers design and build software that effectively meets user expectations. Alan Cooper is the prime proponent of interaction design, a process with similar goals but different methodology. We brought these two visionaries together to compare philosophies, looking for points of consensus—and points of irreconcilable difference.” The difference in perspective between the two is very expressive of the conflicts we have seen between design and agile development processes. Many of the following resources reference this discussion.

Note: the original source for this great interview appears to have been taken over by another entity that is no longer making the article available. I am linking to it in the Internet Archive.

Waterfall Bad, Washing Machine Good
Leisa Reichelt’s IA Summit 07 presentation on the role of IA in the design process identifies the failures of a waterfall design process and argues that an agile, iterative design process is superior. Slide 14 is right on.

User Experience Activities on Agile Development Projects
Recent article by Scott Ambler addresses the gap between Beck and Cooper by identifying a middle ground that requires UX and agile practitioners to learn a bit more about each others’ practices and to adopt some of each.

Holistic Product Design & Development
A series of blog posts by Jeff Patton on agile software development and user centered design. Jeff moderates the agile-usability Yahoo group.

Agile Usability Yahoo! Group
The description for this group provides a good summary of the issues we are tackling here. You are invited to jump into the fray.

Experiences Integrating UED Practices
In this experience report Paul Hodgetts presents his “teams’ struggles to come to grips with the often programming-centric orientation of agile processes, and their ongoing efforts to integrate their UED (user experience design) best practices into the incremental, collaborative world of agile processes.” He provides an overview to the range of UX practices, identifies challenges and discusses practices that he found were particularly useful in integrating UX and and agile development.

Agile User-Centered Design
A post to ID-discuss that presents one designer’s struggle to reconcile agile and user-centered design.

Process Agility and Software Usability: Toward Lightweight Usage-Centered Design
2001 paper by Larry Constantine presents concrete practices for UX that map to agile practices. His approach is simple and pragmatic.

The Challenge of Agile Design
These are slides from a talk given by our friends at Pivotal Labs and Satisfaction.

User Centered Design Round Table
Podcast of a discussion on user-centered design and agile development with Lynn Miller, Jeff Patton and Rebecca Wirfs-Brock.

Originally posted 2007-09-06
Last updated 2008-02-14