iPhone Unit Testing Toolkit

Posted on by in Mobile, Process

As we get in to developing iPhone/Touch applications at Carbon Five, we obviously want to bring over all our best agile practices including test driven development (TDD).

I’ve been bootstrapping with the beta version of the Pragmatic iPhone SDK Development book but was disappointed to find no discussion of unit testing for the iPhone. So I’ve been piecing it together as I go.

GTM

Unit testing in Objective-C is provided by the SenTestingKit framework that installs with XCode in /Developer/Library/Frameworks/. However it is not compatible with the iPhone SDK. The Google Toolbox for Mac provides an implementation of SenTestCase for the iPhone that I am using instead. The instructions provided on that page worked for me with one exception that I included as a comment on that page:

I followed the instructions above but was unable to see any messages in the console. I did not see the behavior described above as:

Your target should now build cleanly, and if you check the build log you should see something like: “Executed 0 tests, with 0 failures (0 unexpected) in 0.001 (0.001) seconds” at the end.

It turned out that I needed to set the base SDK in the inspector for my target to an iPhone simulator even though the Overview showed I was building for simulator debugging.

I created a ‘Unit Tests’ target per the GTM directions to run my tests. Now I am happily running my tests with command-B. One point that could be a problem when getting into continuous integration is that my Unit Tests target reports ‘Build succeeded (1 error)’ when a unit test fails instead of having the build actually fail.

I have not gotten into the “Advanced Stuff” described on that page.

Hamcrest

The SenTestingKit assertion methods are pretty lame. They’re verbose and require that you provide a string message for your expectation (e.g. STAssertEqualStrings(a1, a2, description, …)). We’re using and like Hamcrest on many of our Java projects. I was psyched to see there is an Objective-C implementation OCHamcrest.

There is no download available for Hamcrest Objective-C so I checked it out from SVN and built the framework product in Xcode for OS X 10.5/i386.

I struggled for a while to get the dependency on the hamcrest.framework build product into my project. I ended up with this solution:

  1. Create a ‘Frameworks’ folder in my project root
  2. Copy hamcrest.framework to Frameworks/
  3. In Xcode, right click on ‘Frameworks’ and ‘Add existing framework…”
  4. Browse to the hamcrest.framework folder and add it, making sure to add it to my Unit Tests target only
  5. Right click on the Unit Tests target to add a new Copy Files Build Phase that copies to the Products directory and put that build phase early in the target steps
  6. Add hamcrest.framework to that build phase by dragging from Frameworks
  7. Include Hamcrest in your test class and you’re good to go
#import "GTMSenTestCase.h"
#define HC_SHORTHAND
#import 

@interface ExampleWithAssertThat : SenTestCase
@end

@implementation ExampleWithAssertThat

- (void) testUsingAssertThat
{
    assertThat(@"xx", is(@"xx"));
    assertThat(@"yy", isNot(@"xx"));
    assertThat(@"i like cheese", containsString(@"cheese"));
}

@end

OCMock

iPhone development requires a lot of mucking around with UI framework classes like UITableView, UIApplicationDelegate and so on. These classes have a lot of dependencies and are depended upon all over the place which can make it hard to get them under test.

Mock objects are a great solution to this problem and OCMock provides an implementation for Objective-C. In this case, the OCMock.framework provided in the OCMock binary release worked for me when installed in the same manner as described for Hamcrest above.

This example of testing [UIApplicationDelegate applicationDidFinishLaunching] illustrates its use pretty well. I have a simple test that creates a mock UITableView to asserts a cell is correctly created in [UITableViewController tableView:cellForRowAtIndexPath:]. It looks like:

#import "GTMSenTestCase.h"
#define HC_SHORTHAND
#import 
#import 

#import "SortedStateViewController.h"
#import "Country.h"

@interface SortedStateViewControllerTest : SenTestCase {}

SortedStateViewController *controller;

@end

@implementation SortedStateViewControllerTest

- (void) setUp {
    controller = [[SortedStateViewController alloc] init];
    Country *country = [[Country alloc] init];
    [country addState:@"New York" withPopulation:19306183 andArea:47213];
    [country addState:@"California" withPopulation:36457549     andArea:155959];
    controller.country = country;
    controller.key = @"population";
}

- (void) testPopulatesCell {
    id tableView = [OCMockObject mockForClass:[UITableView class]];
    [[tableView stub] dequeueReusableCellWithIdentifier:[OCMConstraint any]];
    NSIndexPath *indexPath = [NSIndexPath indexPathForRow:(NSUInteger)0
                                                inSection:(NSUInteger)0];

    UITableViewCell *cell = [controller tableView:tableView
                            cellForRowAtIndexPath:indexPath];
    assertThat(@"California", equalTo(cell.text));
}

@end

Continuous Integration

So I’m pretty happy with where this is now. I’m working through the Pragmatic book TDD as I go. It’s also have an ObjectiveCTest that I add assertions to as I play with features of the language.

Missing from this toolkit is a good continuous integration solution.

There is now a recently released Xcode builder for CruiseControl. However, we use Bamboo for CI for our Java and Ruby projects and are considering moving to TeamCity. It looks like there is an Xcode plugin for TeamCity and some attempts with Bamboo.

More on that later.


Feedback

  Comments: 17


  1. We use Hudson for CI and Unitest++ with a custom reporter to do TDD. It all works rather nicely! If you’re interested I could write up a post about it. E-mail me and let me know 🙂


  2. Thanks for all this help. I’m just starting out Objective-C development, coming from a heavily TDD background, so this is invaluable. Thanks again.


  3. The best information i have found exactly here. Keep going Thank you


  4. You also may want to check out UISpec http://code.google.com/p/uispec/

    It’s an open source BDD gui testing framework being developed for the iphone.


  5. This was a great walkthrough – all three libraries are going to become part of my standard project template.

  6. Patrick O’Shaughnessey


    It appears Apple added support for iPhone unit testing in May 2009:

    http://developer.apple.com/IPhone/library/documentation/Xcode/Conceptual/iphone_development/135-Unit_Testing_Applications/unit_testing_applications.html

    Sounds like you’ve found Hamcrest to be superior to SenTestingKit because of its more terse syntax, but the convenience of an out-of-the-box solution is also appealing…


  7. Patrick,

    While you can now run SenTestingKit tests for iPhone apps I have not found that to be a viable solution yet.

    Apple’s logic tests can only run in the iPhone simuator and do not run within an app bundle. Application tests are run within an app bundle but currently run only on an actual device.
    That combination really doesn’t provide what I need for efficient, test driven development. I really want to be able to run my tests as part of my build process, quickly, and still be able to debug them on a device or assert that they interact with the rest of the application correctly. Currently GTM (and more recently GHUnit) seem to allow me to write tests more effectively.

    As a final note, we’re not replacing SenTestingKit with Hamcrest. We’re using Hamcrest’s functions and matchers for our assertions within our tests. Hamcrest and OCMock are useful tools within a test method but it’s GTM or GHUnit that provides the base test class my tests extend and the framework to run those tests.


  8. Hey Jason Citron,

    I would be very interested in a post about how you use Hudson and Unitest++!!

    Thanks.


  9. I would also be interested in a write-up about using hudson with iphone builds. I have hudson running my iphone build targets but I am looking for a simple, reliable mechanism to publish build/test results.


  10. Patrick, OCHamcrest works in conjunction with OCUnit (aka SenTestingKit). Hamcrest is not a testing framework per se, but a means of specifying expected matches. OCMock now supports OCHamcrest matchers.


  11. You might also want to checkout Gorilla Logic’s FoneMonkey, an open source record / playback testing tool for iPhone applications: http://www.gorillalogic.com/fonemonkey


  12. Thanks for highlighting Hamcrest. I just released OCHamcrest 1.0, with full support for iOS development, and much, much easier to set up.


  13. great post but i was wondering if i can user OCHamcrest 1.0 with teamcity??


  14. Hi!

    In your post you talk about “I’m working through the Pragmatic book TDD as I go.” Which book do you mean ?

    With regards,

    Rutger

  15. Static Site Design


    The environment is taking some getting used to and it took me some time to get to a reasonable working environment.

  16. Mobile Pundits


    Great article with very specific points about issues often
    found in iPhone application projects. All of your points should be considered
    when developing a testing strategy and plan. I’d add that a strategy often will consider some tech aspects like performance and unit testing along with disaster recovery, and data migration.
    iPhone Mobile Application Testing

Your feedback