I recently came across a scenario where I needed to make an assertion on an argument passed to an expected method invocation but OCMock did not provide the constraint I needed. I was testing a FileDetailsViewController that pushes a new FileContentsViewController on to the view stack when a button is pressed. I mocked the UINavigationController so that I could add an assertion that pushViewController:animated: is called with the right arguments.

The meat of my test is:

@interface FileDetailsViewControllerTest : SenTestCase {}
    FileDetailsViewController *controller;
    id navigationController;
@end
 
@implementation FileDetailsViewControllerTest
 
-(void) setUp {
    controller = [[FileDetailsViewController alloc] initWithNibName:@"FileDetailsView" 
                                                             bundle:nil];
    navigationController = [OCMockObject mockForClass:[UINavigationController class]];
    controller.navigationController = navigationController;
}
 
-(void) testShowContentLoadsView {
    [[navigationController expect] 
         pushViewController:[OCMConstraint isKindOfClass:[FileContentsViewController class]]
                   animated:YES];
    [controller showContentsButtonWasPressed];
}
 
-(void) tearDown {
    [navigationController verify];
}
 
@end

Note that in this example, I some fanciness is required to assign controller.navigationController = navigationController since it is a readonly property in the UIViewController API. I’ll explain what I did in a later post.

The OCMock feature I was missing is the OCMock.isKindOfClass constraint. So I wrote my own.

Following is the header file that declares the isKindOfClass assertion as a static method in an Objective-C category for OCMock. Categories are kind of like Ruby mixins for Objective-C. You can add static or instance methods to an existing class definition. With this approach I can follow the OCMock pattern of providing static factory methods on the OCMock class for the range of constraints available.

//File: OCMockConstraint+Extensions.h
#import <OCMock/OCMConstraint.h>
 
@interface OCMConstraint (Extensions)
 
+ (id)isKindOfClass:(id)value;
 
@end

Following is the implementation, both of my new constraint and the factory method to create it.

//File: OCMockConstraint+Extensions.m
#import "OCMockConstraint+Extensions.h"
 
@interface OCMKindOfClassConstraint : OCMConstraint
{
    @public
    id testClass;
}
@end
 
@implementation OCMKindOfClassConstraint
 
- (BOOL)evaluate:(id)value
{
	return [value isKindOfClass:testClass];
}
 
@end
 
// Static factory method
@implementation OCMConstraint (Extensions)
 
+ (id)isKindOfClass:(id)value {
	OCMKindOfClassConstraint *constraint = [OCMKindOfClassConstraint constraint];
	constraint->testClass = value;
	return constraint;
}
 
@end

This implementation shows both how to write a custom constraint for OCMock and a use of Objective-C categories to provide a nice API for using it.

In many cases, you’ll want to write a custom constraint that is specific to your test scenario and not appropriate to be shared globally in this manner. In this case you’ll probably follow a similar strategy but implemented in your test class instead of mixed in to OCMockConstraint.

I’ve been bootstrapping with the beta version of the Pragmatic iPhone SDK Development book but was disappointed to find no discussion of unit testing for the iPhone. So I’ve been piecing it together as I go.

GTM

Unit testing in Objective-C is provided by the SenTestingKit framework that installs with XCode in /Developer/Library/Frameworks/. However it is not compatible with the iPhone SDK. The Google Toolbox for Mac provides an implementation of SenTestCase for the iPhone that I am using instead. The instructions provided on that page worked for me with one exception that I included as a comment on that page:

I followed the instructions above but was unable to see any messages in the console. I did not see the behavior described above as:

Your target should now build cleanly, and if you check the build log you should see something like: “Executed 0 tests, with 0 failures (0 unexpected) in 0.001 (0.001) seconds” at the end.

It turned out that I needed to set the base SDK in the inspector for my target to an iPhone simulator even though the Overview showed I was building for simulator debugging.

I created a ‘Unit Tests’ target per the GTM directions to run my tests. Now I am happily running my tests with command-B. One point that could be a problem when getting into continuous integration is that my Unit Tests target reports ‘Build succeeded (1 error)’ when a unit test fails instead of having the build actually fail.

I have not gotten into the “Advanced Stuff” described on that page.

Hamcrest

The SenTestingKit assertion methods are pretty lame. They’re verbose and require that you provide a string message for your expectation (e.g. STAssertEqualStrings(a1, a2, description, …)). We’re using and like Hamcrest on many of our Java projects. I was psyched to see there is an Objective-C implementation.

There is no download available for Hamcrest Objective-C so I checked it out from SVN and built the framework product in Xcode for OS X 10.5/i386.

I struggled for a while to get the dependency on the hamcrest.framework build product into my project. I ended up with this solution:

1. Create a ‘Frameworks’ folder in my project root
2. Copy hamcrest.framework to Frameworks/
3. In Xcode, right click on ‘Frameworks’ and ‘Add existing framework…”
4. Browse to the hamcrest.framework folder and add it, making sure to add it to my Unit Tests target only
5. Right click on the Unit Tests target to add a new Copy Files Build Phase that copies to the Products directory and put that build phase early in the target steps
6. Add hamcrest.framework to that build phase by dragging from Frameworks
7. Include Hamcrest in your test class and you’re good to go

#import "GTMSenTestCase.h"
#define HC_SHORTHAND
#import <hamcrest/hamcrest.h>
 
@interface ExampleWithAssertThat : SenTestCase
@end
 
@implementation ExampleWithAssertThat
 
- (void) testUsingAssertThat
{
    assertThat(@"xx", is(@"xx"));
    assertThat(@"yy", isNot(@"xx"));
    assertThat(@"i like cheese", containsString(@"cheese"));
}
 
@end

OCMock

iPhone development requires a lot of mucking around with UI framework classes like UITableView, UIApplicationDelegate and so on. These classes have a lot of dependencies and are depended upon all over the place which can make it hard to get them under test.

Mock objects are a great solution to this problem and OCMock provides an implementation for Objective-C. In this case, the OCMock.framework provided in the OCMock binary release worked for me when installed in the same manner as described for Hamcrest above.

This example of testing [UIApplicationDelegate applicationDidFinishLaunching] illustrates its use pretty well. I have a simple test that creates a mock UITableView to asserts a cell is correctly created in [UITableViewController tableView:cellForRowAtIndexPath:]. It looks like:

#import "GTMSenTestCase.h"
#define HC_SHORTHAND
#import <hamcrest/hamcrest.h>
#import <OCMock/OCMock.h>
 
#import "SortedStateViewController.h"
#import "Country.h"
 
@interface SortedStateViewControllerTest : SenTestCase {}
 
SortedStateViewController *controller;
 
@end
 
@implementation SortedStateViewControllerTest
 
- (void) setUp {
    controller = [[SortedStateViewController alloc] init];
    Country *country = [[Country alloc] init];
    [country addState:@"New York" withPopulation:19306183 andArea:47213];
    [country addState:@"California" withPopulation:36457549     andArea:155959];
    controller.country = country;
    controller.key = @"population";
}
 
- (void) testPopulatesCell {
    id tableView = [OCMockObject mockForClass:[UITableView class]];
    [[tableView stub] dequeueReusableCellWithIdentifier:[OCMConstraint any]];
    NSIndexPath *indexPath = [NSIndexPath indexPathForRow:(NSUInteger)0 
                                                inSection:(NSUInteger)0];
 
    UITableViewCell *cell = [controller tableView:tableView 
                            cellForRowAtIndexPath:indexPath];
    assertThat(@"California", equalTo(cell.text));
}
 
@end

Continuous Integration

So I’m pretty happy with where this is now. I’m working through the Pragmatic book TDD as I go. It’s also have an ObjectiveCTest that I add assertions to as I play with features of the language.

Missing from this toolkit is a good continuous integration solution.

There is now a recently released Xcode builder for CruiseControl. However, we use Bamboo for CI for our Java and Ruby projects and are considering moving to TeamCity. It looks like there is an Xcode plugin for TeamCity and some attempts with Bamboo.

More on that later.

Maybe it’s because we have a special opportunity to optimize the hell out of our development process, or the fact that we’re all productivity junkies; regardless of the reason, we religiously embrace the tenant “Make the things you do often fast and easy”. It’s almost embarrassing to suggest that others don’t also subscribe to this simple notion, but — brace yourself — many do not. On a project that has history, not everyone has been there for every decision. In fact, many developers are at least relatively new and it’s somewhat customary to have a “it must be this way for a reason” attitude. After all, who would deliberately make something cumbersome without good reason?

When we start working on one of these projects, we dedicate time to do some serious spring cleaning and tackle the things that will cost us the most in terms of pain and productivity. The whole development team gets psyched about where we end up as it’s a significant improvement. Projects with a history usually have a fair bit of low-hanging fruit. Let’s discuss some of the things we see regularly.

Build Systems

Apparently few people like working on build scripts and when they do they have a habit of lowering their standards for quality of work. That’s obviously not literally true, but sometimes it seems that way. We’ve seen a number of beastly build systems that are slow because they’re doing things that aren’t necessary (extraneous jaring, copying, code generating, etc), they’re brittle and expensive to maintain, and full of dead code and duplicate target definitions… and they’re run many times every single work day. It’s true that most developers may be compiling code from their IDE and thus bypassing the command-line build, but it’s still run on the build server, by ops folks, and even by developers when they’re debugging why something works from the IDE but busts on the build server.

Guidelines for simplifying the build

• Distill the build process down to the fastest, simplest steps that are necessary.
• Eliminate duplicate and no-longer used dependencies; these files are being copied around and bundled for no reason (I’ve seen over 15 megs of unnecessary dependencies before).
• When a project is split into multiple modules (and it should if it’s more than a few thousand lines), modules should be built in a consistent fashion using targets that are shared across modules.
• Look for exceptions. When you see something special happening for a particular file type, file name, or modules, ask yourself why. Ask again. Strive to eliminate these special cases when possible, even if they seem trivial.
• Build a single deployable (or deployables) for all environments by eliminating environment-specific build code and externalizing application configuration. (Use Spring? Check out this post on externalizing configuration with Spring.)
• Look for unnecessary code generation steps; if generated code changes once a year then check it in and make regenerating a manual step.
• If you generate code coverage data, make sure that it’s only created when it’s needed (e.g. a nightly build on the build server), not on every build.

So, we use Maven 2 for all of our Java projects. For sure, it has its share of rough edges (most of which are being fixed at a reasonable rate). But it recommends some very sound conventions and doesn’t provide any scripting functionality, so it’s harder to hack it to do anything too unorthodox (please don’t use the antrun plugin unless as an incremental step when moving from Ant to Maven). When you play ball by the Maven rules you’ll find your build much simpler and easier to maintain. It’s likely you’ll notice other emergent benefits to boot. For example, once you migrate to Maven you eliminate duplicate build configuration (both your command line build tool and IDE know how to compile your app — remember the DRY principle). IDEA, Eclipse (via m2eclipse), and NetBeans all support importing from and synchronize with Maven.

Some people use Buildr or Ant + Ivy, but either they don’t have the breadth of use (Buildr) or are more susceptible to writing nasty, unmaintainable build code (Ant). That’s why we use Maven.

Compile > Deploy > Make Changes > Deploy Development Cycle

Possibly more important than a simple and easy build, developers must be able to go through the compile, deploy, make changes, deploy cycle FAST (note that the compile, run tests, make changes, run tests cycle is also very important).

I remember working on an embedded system in 1998: a complex radio communications routing application written in C++ and deployed to custom hardware running the real-time operating system PSOS. The build and deploy cycle took about 30 minutes and there were only 10 hardware instances for 60 engineers; you had to sign up for time slot on real hardware. It was the epitome of unproductive as far as development environments go (and don’t even ask about debugging!). You’d think such things were completely in the past (luckily they mostly are), but they’re not completely. In the last 2 years I’ve seen applications that take 15 minutes to deploy.

It’s a drag when developers have to wait for these things to happen and it can totally destroy one’s rhythm, keeping developers from getting into the zone. What’s worse, it’s completely unnecessary with modern tools.

General Suggestions

• Don’t drop down to the command-line; compile and deploy from your IDE (the IDE is your friend – master it).
• Minimize steps for deploying changes to a running app:
  • Your IDE may support building on frame deactivation (IDEA does); check it out.
  • Look into the maven-jetty-plugin if you use Maven.
  • Run in debug mode so that code can be hot-swapped or invest in JavaRebel, which allows all sorts of code changes to take place without redeploying your application.
  • Deploy your application in exploded form; bundling a war or ear incurs unnecessary IO overhead.
• Host your database either locally on your workstation or on a beefy database server on the same LAN. Remote databases are generally many times slower due to latency, even over fast connections.
• Use JBoss? Consider migrating to Jetty or Tomcat. If that’s impossible, use the most stripped down profile (minimal, default, or all) that has what you need, or better yet, create a custom one which includes only what you need.
• Minimize the amount of data needed in the database to run the application. The same goes for running tests: do whatever it takes to run your tests against an empty (or very close to) schema. Check out the Carbon Five DB Migration Project.
• Don’t skimp on developer hardware. Buying the very fastest CPU isn’t going to be worth it, so aim for one or two models down from the fastest. Buy the fastest hard disk you can since development is generally IO bound (WD Raptor and the new VelociRaptor are awesome, consider SSDs if your coffers run deep). Avoid older CPU architectures (Pentium D), even when the clock speed (GHz) is faster. Lastly, don’t be shy with memory; 4GB isn’t too much for a developer machine.

In addition to these general recommendations, each individual application will have its own specific sources of inefficiency. Many real world applications depend on services provided by application servers and/or a commercial products: message queues (JMS, ActiveMQ, etc), distributed caches (memcached, coherence, etc), enterprise service buses, job schedulers, work flow engines, etc. It’s important that these services don’t get in the way of developing fast. Some of them can be run in a light-weight development mode. If you need to use one of these potentially heavyweight solutions, invest the time to minimize or eliminate any adverse effects to the development cycle.

What Else?

Some of the best improvements have nothing to do with the technical side of software development. Take a step back and look at what else is happening (or not happening) each day. There may be meetings which can be time boxed, consolidated, or eliminated all together. Take a look at collaboration between engineers, product managers, testers, support and operations. How long are developers waiting to have requirements-clarifying questions answered? Ask your whole team where they think things can be improved. Ask for feedback on a regular basis and allow it to help drive these improvements.

Survey of other activities that should be fast and easy

• Running the automated test suite – < 10 minutes
• Getting build results from continuous integration server – < 10 minutes
• Pushing a build to staging/acceptance server – One click build and deploy
• Create a new instance of a minimal database instance – Carbon Five DB Migration Project
• Recreate production state in development for debugging
• Story approval/acceptance – Continuous acceptance
• Meetings – Timebox, Consolidate, Eliminate
• Configure a new developer machine – Strive for zero configuration

Value Simplicity

Any intelligent fool can make things bigger, more complex and more violent. It takes a touch of genius and a lot of courage to move in the opposite direction. -Albert Einstein

There’s a theme underlying most of the solutions to these problems: simplicity. Complex systems don’t become complex and crufty overnight, they get that way one small step at a time. With each change to a system it’s important to recognize that the change will either add complexity or remove it. Complexity has a cost and it’s not to be taken lightly; make sure the benefit to each of the decisions that add complexity is worth that cost.

Conclusion

Making the things that people do often fast and easy can pay off geometrically as all developers benefit and regain a little more of their day (and sanity). In the end, it’s not just about shaving off seconds or minutes, though that’s a huge part of it; it’s about creating a development environment that lets the team do what’s really important: write awesome code to solve real problems. When the team dynamics, technical environment, and process are tuned just right, the overall benefit is greater than the sum of its parts.

Where have you seen changes in infrastructure, software, or process that’s resulted in a significant productivity bump?

This is the first of a series of articles about improving the quality of existing software and the processes that produce it.

Why Not Rewrite

We’ve all encountered software that we would rather just rewrite than try to fix. The problem may be poor design, poor implementation, lack of tests, bugs or any of a host of other software ills. We see ahead of us the pain of working with a poor quality system and the risks to schedule and quality of end product it creates.

At Carbon Five, we have participated in many successful projects to rewrite applications for our clients. Sometimes starting with a clean slate and all the lessons learned from a previous effort is exactly the right approach. Often it is not.

In our experience the greatest obstacle to a rewrite effort is that the existing system is the only accurate record of the requirements for the system. Applications evolve over years to meet the changing needs of a organization and its users. Many small decisions are made and captured in the functioning application only. Rewriting to reproduce the features of an existing system can be a very difficult effort to define and scope.

Usually the troubled application will continue to be use while a rewrite effort is underway. Often support, maintenance and feature development will need to continue on that application even while a rewrite is under way. A rewrite will compete for development resources and will be chasing new feature development on the existing application.

Not everything in the troubled application is rotten. There are often good pieces, or at least components that are reliable and well understood. It would be great to not have to rewrite them.

How can you be sure that a rewritten application will be so much better than the one it is replacing? All too often, the source of poor software quality is poor process and practices. Unless you fix those problems it is not worth embarking on a new software development effort.

How to Rescue

Sometimes a decision to rewrite is made because rescuing an application seems too daunting. Where do you start? How long will it take? Here are some high level thoughts from our experiences with our customers. We will get in to more detail on specifics in later articles.

Set Short Term Goals

Be sure to set demonstrable short term goals with meaningful names. When your job is to make the system “better” your job is never done. You need discrete goals with names that you can finish, demonstrate, release and feel good about before moving on. Easily communicated goals help to show management that the effort is progressing successfully and help your team build confidence in an often daunting task.

On one of our projects we set a goal of having a unit test in place for a component in our service layer. “Service Test” would be a fine name for that goal. In a system with no testing support, this was a big task that once completed opened the door for much easier testing.

Don’t Let “Refactor” Become a Dirty Word

A lot of the work you will do when rescuing an application can be characterized as refactoring:

Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.
Martin Fowler, http://www.refactoring.com

Don’t let “refactor” become a dirty word. It’s too easy to say, “Yesterday I worked on refactoring our persistence layer. Today I’m going to keep working on that.” Again, come up with a name or description for the work you are doing that enables others to understand what you are doing and for you to describe progress through that task. Refactoring is one activity of many that you will engage in during this process. Avoid using “refactor” in the names of your short term goals.

Introduce Testing

A common attribute of many struggling applications is conspicuous lack of automated testing and the design and infrastructure to support it. We have found that the many changes required to support and write automated unit and integration tests are exactly the changes an application needs to get healthy. They include:

• Automating and streamlining build systems
• Decoupling application components
• Minimizing dependencies on application server features
• Building test data sets
• Continuous integration
• Building a team interest in quality

Each of these items is challenging to implement. You can’t do them all at once yet they are dependent on each other. Fortunately there are good open tools and practices available that provide a helpful bootstrap including Maven for convention-based builds, EasyMock to test code with lots of dependencies, Spring for component management outside of an application server, and CruiseControl for continuous integration (not to ignore the excellent almost-free options like TeamCity and Bamboo).

Introducing testing to a legacy application with no support for it is a daunting challenge and slow going at first. We’ve had good success in the past by first cleaning up the build and adding the ability to run tests, then finding a place in the application to break apart components and wedge a first test in. With a running test, you have something to automate so you can get continuous integration running. You also have a lot more insight into the architectural changes that will both improve the health of the application and make it easier to add more tests.

We have a lot more to say about this topic and specific experiences to share in coming articles.

A Culture of Quality

Another common attribute of a struggling application is that the team maintaining and developing the application is in continuous firefighting mode. They’ve lost hope of making fundamental improvements to their system and on a daily basis are fixing emergency bugs and hacking in features with their fingers crossed that they are not breaking something else.

Teams working in this mode collaborate poorly with product management. Product management feels that the developers are resistant to new features. Getting releases out is painful and unpredictable because defect rates are high.

These teams also collaborate poorly with each other. Developers tend to become isolated from each other as sole maintainers of one application component or another. They do not talk about how to make things better and become resigned to living with the status quo.

To be successful in a rescue mission or even a rewrite, you have to turn the firefighting culture into one where developers value quality and work for it daily. They should be excited to make things better and be engaging each other with ideas and practices to get there.

Again, this can be a very difficult effort. Sometimes it requires dramatic measures. In our experience this includes:

• Changing a workspace to remove barriers to casual conversation
• Relocating developers to the same physical location
• Hiring new blood and firing those resistant to change
• Pair programming
• Book clubs and study groups

These changes are often the most painful to make. As a consulting company Carbon Five can advocate for these changes, teach pair programming and run a study group, but the real changes have to come from within. We have seen the most success in making these changes when there is an commitment to improve from the business and champions of this effort in management.

The post is the first of a series on user experience (UX) design practices for agile software development teams. In an upcoming project with our partner Hot Studios we will be exploring new ways to integrate their user experience design process with our software development process. We’ll share some learnings from that, experiences from the past and new ideas for the future.

First we have a growing reading list of existing resources on the topic that I am posting here for you. If you have other resources for us, please share.

User Research as a Commodity
Added 2008-02-14
Alex at Pivotal Labs noted this post from a user experience manager at Kaiser Permanente who describes a strategy for including regular user testing in an agile development process. The strategy is attractive because it fits with a key characteristic of agile where simple practices performed regularly produce high level complex results.

Kent Beck v. Alan Cooper
This is a 2002 article with the following abstract “Kent Beck is known as the father of ‘extreme programming,’ a process created to help developers design and build software that effectively meets user expectations. Alan Cooper is the prime proponent of interaction design, a process with similar goals but different methodology. We brought these two visionaries together to compare philosophies, looking for points of consensus—and points of irreconcilable difference.” The difference in perspective between the two is very expressive of the conflicts we have seen between design and agile development processes. Many of the following resources reference this discussion.

Note: the original source for this great interview appears to have been taken over by another entity that is no longer making the article available. I am linking to it in the Internet Archive.

Waterfall Bad, Washing Machine Good
Leisa Reichelt’s IA Summit 07 presentation on the role of IA in the design process identifies the failures of a waterfall design process and argues that an agile, iterative design process is superior. Slide 14 is right on.

User Experience Activities on Agile Development Projects
Recent article by Scott Ambler addresses the gap between Beck and Cooper by identifying a middle ground that requires UX and agile practitioners to learn a bit more about each others’ practices and to adopt some of each.

Holistic Product Design & Development
A series of blog posts by Jeff Patton on agile software development and user centered design. Jeff moderates the agile-usability Yahoo group.

Agile Usability Yahoo! Group
The description for this group provides a good summary of the issues we are tackling here. You are invited to jump into the fray.

Experiences Integrating UED Practices
In this experience report Paul Hodgetts presents his “teams’ struggles to come to grips with the often programming-centric orientation of agile processes, and their ongoing efforts to integrate their UED (user experience design) best practices into the incremental, collaborative world of agile processes.” He provides an overview to the range of UX practices, identifies challenges and discusses practices that he found were particularly useful in integrating UX and and agile development.

Agile User-Centered Design
A post to ID-discuss that presents one designer’s struggle to reconcile agile and user-centered design.

Process Agility and Software Usability: Toward Lightweight Usage-Centered Design
2001 paper by Larry Constantine presents concrete practices for UX that map to agile practices. His approach is simple and pragmatic.

The Challenge of Agile Design
These are slides from a talk given by our friends at Pivotal Labs and Satisfaction.

User Centered Design Round Table
Podcast of a discussion on user-centered design and agile development with Lynn Miller, Jeff Patton and Rebecca Wirfs-Brock.

Originally posted 2007-09-06
Last updated 2008-02-14