The post is the first of a series on user experience (UX) design practices for agile software development teams. In an upcoming project with our partner Hot Studios we will be exploring new ways to integrate their user experience design process with our software development process. We’ll share some learnings from that, experiences from the past and new ideas for the future.

First we have a growing reading list of existing resources on the topic that I am posting here for you. If you have other resources for us, please share.

User Research as a Commodity
Added 2008-02-14
Alex at Pivotal Labs noted this post from a user experience manager at Kaiser Permanente who describes a strategy for including regular user testing in an agile development process. The strategy is attractive because it fits with a key characteristic of agile where simple practices performed regularly produce high level complex results.

Kent Beck v. Alan Cooper
This is a 2002 article with the following abstract “Kent Beck is known as the father of ‘extreme programming,’ a process created to help developers design and build software that effectively meets user expectations. Alan Cooper is the prime proponent of interaction design, a process with similar goals but different methodology. We brought these two visionaries together to compare philosophies, looking for points of consensus—and points of irreconcilable difference.” The difference in perspective between the two is very expressive of the conflicts we have seen between design and agile development processes. Many of the following resources reference this discussion.

Note: the original source for this great interview appears to have been taken over by another entity that is no longer making the article available. I am linking to it in the Internet Archive.

Waterfall Bad, Washing Machine Good
Leisa Reichelt’s IA Summit 07 presentation on the role of IA in the design process identifies the failures of a waterfall design process and argues that an agile, iterative design process is superior. Slide 14 is right on.

User Experience Activities on Agile Development Projects
Recent article by Scott Ambler addresses the gap between Beck and Cooper by identifying a middle ground that requires UX and agile practitioners to learn a bit more about each others’ practices and to adopt some of each.

Holistic Product Design & Development
A series of blog posts by Jeff Patton on agile software development and user centered design. Jeff moderates the agile-usability Yahoo group.

Agile Usability Yahoo! Group
The description for this group provides a good summary of the issues we are tackling here. You are invited to jump into the fray.

Experiences Integrating UED Practices
In this experience report Paul Hodgetts presents his “teams’ struggles to come to grips with the often programming-centric orientation of agile processes, and their ongoing efforts to integrate their UED (user experience design) best practices into the incremental, collaborative world of agile processes.” He provides an overview to the range of UX practices, identifies challenges and discusses practices that he found were particularly useful in integrating UX and and agile development.

Agile User-Centered Design
A post to ID-discuss that presents one designer’s struggle to reconcile agile and user-centered design.

Process Agility and Software Usability: Toward Lightweight Usage-Centered Design
2001 paper by Larry Constantine presents concrete practices for UX that map to agile practices. His approach is simple and pragmatic.

The Challenge of Agile Design
These are slides from a talk given by our friends at Pivotal Labs and Satisfaction.

User Centered Design Round Table
Podcast of a discussion on user-centered design and agile development with Lynn Miller, Jeff Patton and Rebecca Wirfs-Brock.

Originally posted 2007-09-06
Last updated 2008-02-14

This release appears to be the same version of Flash Remoting that is distributed with Macromedia Flex. It bundles the Carbon Five ASTranslator library for JavaBean-style serialization and allows the use of custom adapters like our SpringBeanAdapter for using Flash Remoting with the Spring Framework.

We have documented and released this implementation on our Sourceforge Open Source site.

SpringBeanAdapter is designed to be used with Macromedia Flash Remoting MX on any J2EE application server.
This adapter only works with versions of Flash Remoting that allow registering custom adapters, specifically, the version provided with Macromedia Flex.

SpringBeanAdapter is supported by Carbon Five.

• Overview
• Features
• Download
• Prerequisites
• Install
• Usage
• Error Handling

Overview

Flash Remoting comes with a number of adapters for locating remote services implemented as Java classes, EJBs, servlets, and JMX Beans.

The Spring Framework is a popular open source framework for managing
enterprise Java application components.

SpringBeanAdapter provides the ability to locate Flash Remoting services in a Spring application context.

Features

• Locate Remoting services by bean name in Spring application context
• Customizable error handling of server-side exceptions

Download

The latest release of SpringBeanAdapter is available for download from SourceForge.net. For the adventurous, you can check out the latest changes from CVS.

Prerequisites

SpringBeanAdapter depends on:

• The Spring Framework – home page
• Macromedia Flash Remoting (Flex version) – Flex home page

Install

To install SpringBeanAdapter you must unzip springadapter-x.x.zip and copy the following jars to your web application’s WEB-INF/lib/ directory:

• lib/springadapter-x.x.jar

Configure Flash Remoting to use SpringBeanAdapter by editing your Flex Flash Remoting configuration file located in WEB-INF/flex/gateway-config.xml.

<service-adapters>
  <adapter type="stateless-class">com.carbonfive.flash.spring.SpringBeanAdapter</adapter>
</service-adapters>

Usage

SpringBeanAdapter handles services named with a ‘spring://’ prefix. We recommend using named remote services in Flex by adding services to your Flex configuration in WEB-INF/flex/flex-config.xml:

The following is just a sample from a flex-config.xml file:

<remote-objects>
  <named>
    <object name="LocalServiceName">
      <source>spring://RemoteServiceName</source>
      <type>stateless-class</type>
      <allow-unnamed-access>false</allow-unnamed-access>
    </object>
  </named>
</remote-objects>

With this configuration, you can then use remote object services from your Flex application with:

<mx:RemoteObject id="serviceName" named="LocalServiceName"/>

Error Handling

SpreadBeanAdapter provides a nice feature for developers who want control over how errors that happen on the server are sent to the Flash client.

In the standard adapters provided by Macromedia, server errors are converted to a class, flashgateway.GatewayException, that contains the same properties as the fault object received by the Flash client:

• type
• code
• message
• details

SpringBeanAdapter allows you to provide a factory class to produce your own GatewayExceptions when an error occurs on the server. To use this feature, register a bean that implements the interface com.carbonfive.flash.spring.RemotingExceptionFactory with the name ‘remotingExceptionFactory’ in your Spring application context. SpringBeanAdapter will look for and use this class if found.

An example implementation of RemotingExceptionFactory looks like:

public class ExampleExceptionFactory
  implements java.io.Serializable, RemotingExceptionFactory
{
  public static String CUSTOM_CODE = "Server.Custom";
 
  public void throwException(Throwable t) throws Exception
  {
    if (t instanceof SecurityException)
    {
      GatewayException e = new GatewayException(t.getMessage());
      e.setRootCause(t);
      e.setCode(GatewayException.SERVER_AUTHORIZATION);
      throw e;
    }
    else if (t instanceof IndexOutOfBoundsException)
    {
      GatewayException e = new GatewayException(t.getMessage());
      e.setRootCause(t);
      e.setCode(CUSTOM_CODE);
      throw e;
    }
 
    if (t instanceof Exception) throw (Exception) t;
    else if (t instanceof Error) throw (Error) t;
  }
}

If you do not convert the exception to a GatewayException in your implementation of RemotingExceptionFactory, it will be handled by the standard Flash Remoting exception handling process.

One of the first things I noticed when I began investigating the performance of remoted lists is that the delay between the time the load starts and time the data becomes accessible can be broken in to two parts. The first is the time between the remote method call and the result handler of that call. During this time the mouse icon is a spinning clock, but the application is still responsive. Within the result handler all the data from the list seems to be available. The second part of the load happens after the result handler. During this time, the application becomes unresponsive. It is only after this time that any visual changes to the application caused by the new data will appear (though the delay exists even if there is no change to the display). It is this time that exhibits the strange behavior. Since I can’t explain what the application is doing during this time, I’ve dubbed it the mystery time.

My test case involves a simple remote object called TestRemote:

package test;
 
import java.io.*;
import java.util.*;
 
public class TestRemote implements Serializable
{
  public List getObjects(int number, int mapSize, int references)
  {
    List data = new ArrayList();
    Random rnd = new Random();
 
    TestParent parent = new TestParent(Integer.toString(rnd.nextInt()), rnd.nextInt(), "Test Object " + rnd.nextInt(), testMap(mapSize, rnd));
    TestParent parent2 = new TestParent(Integer.toString(rnd.nextInt()), rnd.nextInt(), "Test Object " + rnd.nextInt(), testMap(mapSize, rnd));
    for (int i = 0; i < number; i++)
      data.add(new TestRemotingObject(Integer.toString(rnd.nextInt()), rnd.nextInt(),
                                      "Test Object " + rnd.nextInt(), testMap(mapSize, rnd),
                                      (references > 0 ? parent : null), (references > 1 ? parent2 : null)));
    return data;
  }
 
  private Map testMap(int mapSize, Random rnd)
  {
    Map testData = new HashMap();
    for (int j = 0; j < mapSize; j++)
      testData.put("key" + rnd.nextInt(), "value" + rnd.nextInt());
    return testData;
  }
 
  public static class TestRemotingObject extends TestParent
  {
    public TestParent parent = null;
    public TestParent parent2 = null;
 
    public TestRemotingObject(String id, int number, String name, Map testData, TestParent parent, TestParent parent2)
    {
      super(id, number, name, testData);
      this.parent = parent;
      this.parent2 = parent2;
    }
  }
 
  public static class TestParent
  {
    public String id;
    public int number;
    public String name;
    public Map testData;
 
    public TestParent(String id, int number, String name, Map testData)
    {
      this.id = id;
      this.number = number;
      this.name = name;
      this.testData = testData;
    }
  }
}

and an MXML file with the loading timing and charting logic built in:

<mx:Application xmlns:mx="http://www.macromedia.com/2003/mxml" xmlns:tops="*" pageTitle="Test Peformance" backgroundColor="white">
  <mx:Script><![CDATA[
import mx.core.*;
import mx.utils.*;
 
var MAX_ITEM_COUNT = 1000;
var ITEM_COUNT_INCREMENT = 20;
var MAP_SIZE = 10;
var REFERENCES = 0;
 
var timesData : Array = [];
var loadStart : Date;
var loadMiddle : Date;
var itemCount : Number = 0;
 
public function loadData()
{
  if ( loadStart != null )
  {
    var timeNow = new Date();
    timesData.push( { items:itemCount, loadTime:(loadMiddle.getTime() - loadStart.getTime()), mysteryTime:(timeNow.getTime()-loadMiddle.getTime()), cumulativeTime:(timeNow.getTime()-loadStart.getTime()) } );
    timesChart.executeBindings();
  }
 
  itemCount += ITEM_COUNT_INCREMENT;
  if ( itemCount > MAX_ITEM_COUNT ) return;
 
  loadStart = new Date();
  var contentCall = remoteService.getObjects( itemCount, MAP_SIZE, REFERENCES );
  contentCall.resultHandler = Delegate.create(this,function( result )
  {
    trace(result[itemCount - 1].name);
    loadMiddle = new Date();
    doLater(this, "loadData");
  });
}
]]></mx:Script>
  <mx:RemoteObject id="remoteService" named="TestRemote" showBusyCursor="true" concurrency="multiple" result="event.call.resultHandler( event.result );"/>
  <mx:Button label="load" click="loadData();"/>
  <mx:LineChart id="timesChart" marginLeft="5" height="100%" width="100%"
                marginRight="5" dataProvider="{timesData}" showDataTips="true">
    <mx:horizontalAxis><mx:CategoryAxis dataProvider="{timesData}" categoryField="items"/></mx:horizontalAxis>
    <mx:series>
      <mx:Array>
        <mx:LineSeries yField="loadTime" form="curve" name="Load Time"/>
        <mx:LineSeries yField="mysteryTime" form="curve" name="Mystery Time"/>
        <mx:LineSeries yField="cumulativeTime" form="curve" name="Cumulative Time"/>
      </mx:Array>
    </mx:series>
  </mx:LineChart>
  <mx:Legend dataProvider="timesChart" direction="horizontal"/>
</mx:Application>

The loadData() function of this MXML file is a little strange. The mystery time only occurs after the remote call’s result handler has completed execution, so we need to time between the end of the method and then next time we can execute code. It turns out that Flash’s doLater() method makes this easy. It executes the given method during the next frame of the flash movie, and the mystery time must complete before the next frame is drawn/executed. So the loadData() function records the time since the last call started, increments the item count (and stops if its reached the limit), calls the remote method, and assingns a result handler that will restart the loop by calling loadData() in a doLater() function.

Armed with this code, we can begin analysing the mystery time. I ran many tests with different item counts, map sizes, and object references, but this graph seems to say most everything that I can say with any confidence about the problem. The load time is the time between the remote call and the result handler. It increases linearly with list size as we’d expect (bigger list size = bigger download = increased time). Remember that the result of the remote call is fully available at the end of the load time, so this curve represents the request to the server, the server call itself, the serialization to a remoting response, the streaming of that response, and (it appears) the deserialization of that response into Actionscript objects.

The mystery time is the time between the call to the result handler and the display refresh (measured as the time between the result handler and the next frame). It’s a little difficult to see, but there’s a slight increase in slope at around 15 to 30 items. You can also see that after around 60 items, the mystery time provides the majority of the delay between the the remote request and the response to the user.

In this graph, the mystery time has a spike at around 188 items. That’s common, the peaks sometimes appear in the load time as well. It could be many things, including garbage collection in the flash player, garbage collection in the app server (since I’m running it on the same machine), or an entirely separate process. At any rate, I’m ignoring the spikes.

The most interesting curve is the time per 100 items. If the cumumlative time were linear, this curve would be a horizontal line. As you can see, it starts high and quickly decreases to a minimum. This accounts for the overhead involved in making many many small requests as opposed to a single larger one. The minimum of this curve it the ideal (or at least fastest) paging size. Unfortunately, it isn’t consistent across tests, even with the same parameters. It ranges from about 20 to 60, but is usually on the low side. Between 50 and 100 items, the curve rises quickly, meaning the performance for the high side of 100 items is much worse than on the low side. It’s not so easy to see in this graph, but the curve continues to rise gradually after 100, but slope seems to diminish as the item count continues to rise.

A curiosity of the mystery time, is that it seems to be nearly invariant over the size of the objects in the list. In this test, the item count was kept constant at 100 as the size of the map in each object was incremented. The load time increases linearly as we’d expect, but the mystery time increases only slightly between 0 and 25 map entries, and then remains constant. I’ve observed this for different types of objects as well.

While the mystery time does not depend on the size of the objects, it does seem to depend on the structure of the object graph. Specifically, references to parent objects (i.e. each object in the list referencing a single instance of a container object) seems to increase the mystery time well out of proportion to the actual number of items (N+1) when the item count is high (>> 100). Since we’ve already established that the page size needs to be < 100, I'm not going to elaborate on this oddity.

The conclusions are: 1) the mystery time takes a significant if not dominant proportion of the total request time, 2) The mystery time increases non-linearly with respect to list size and remains constant with respect to object instance size, and 3) optimum paging size seems to be between 20 and 50. I’m at a loss to explain what the mystery time is. It’s doesn’t seem to be deserialization because the content is available before the mystery time starts. It may be garbage collection, but it seems far to predictable for any garbage collection scheme I know of. At any rate, I’d love to hear some theories about what’s going on here.