Multithreaded Testing

Christian Nelson ·

Every now and then you’ll work on something that needs to handle requests from multiple concurrent threads in a special way. I say “special way” because in a web application, everything needs to handle being executed concurrently and there are a slew of techniques used to handle this (prototypes, thread locals, stateless services, etc). Here’s an example of what I mean by “special”…

On my current project, we have a queue of articles that need human-user attention (i.e. editorial moderation). Each article must be doled out to only one moderator and there are multiple instances of the web application servicing requests in the cluster. Imagine tens of thousands of articles per day and a team of moderators churning through them. We can’t rely on Java synchronization because it only works within the JVM instance, not across server instances.

The simplified version of the service interface we’re working on looks like this: – Example service interface

public interface ArticleService
    Article findNextArticleForModeration();

What makes this interesting is that we must ensure that the service doesn’t hand out the same Article to more than one user. This is impossible to assert using a single thread. We’ve all been told that multiple threads and automated testing don’t mix. It’s generally true and should be avoided if at all possible, but in some cases it’s the only way we can truly assert specific behavior. I’ve found a pretty simple way to do this type of testing in a reliable, consistent, and non-disruptive manner. Despite the fact that the technique leverages Java 1.5 built-in concurrency utilities, most of the engineers who have seen it are surprised and weren’t aware that such testing was so easy to implement.

Given the above service interface, here’s a test that will assert that no single article is given out to more than one invoker of the method findNextArticleForModeration(). The scenario we’re simulating is 10 users feverishly moderating a queue of 250 articles as quickly as possible. – Test to invoke service concurrently

public void findNextArticleForModerationStressTest() throws Exception
    final int ARTICLE_COUNT = 250;
    final int THREAD_COUNT = 10;

    // Create test data and callable tasks
    Set<article> testArticles = new HashSet<article>();

    Collection&lt;callable<article>&gt; tasks = new ArrayList&lt;callable<article>&gt;();
    for (int i = 0; i &lt; ARTICLE_COUNT; i++)
        // Test data
        testArticles.add(new Article());

        // Tasks - each task makes exactly one service invocation.
        tasks.add(new Callable<article>()
            public Article call() throws Exception
                return articleService.findNextArticleForModeration();

    // Execute tasks
    ExecutorService executorService = Executors.newFixedThreadPool(THREAD_COUNT);
    // invokeAll() blocks until all tasks have run...
    List&lt;future<article>&gt; futures = executorService.invokeAll(tasks);
    assertThat(futures.size(), is(ARTICLE_COUNT));

    // Assertions
    Set articleIds = new HashSet(ARTICLE_COUNT);
    for (Future<article> future : futures)
        // get() will throw an exception if an exception was thrown by the service.
        Article article = future.get();
        // Did we get an article?
        assertThat(article, not(nullValue()));
        // Did the service lock the article before returning?
        assertThat(article.isLocked(), is(true));
        // Is the article id unique (see Set.add() javadoc)?
        assertThat(articleIds.add(article.getId()), is(true));
    // Did we get the right number of article ids?
    assertThat(articleIds.size(), is(ARTICLE_COUNT));

The test starts off by creating 250 test articles to be moderated. It also creates 250 ‘tasks’, each designed to make a single service invocation of findNextArticleForModeration(). The real magic happens in Executors.newFixedThreadPool() and executorService.invokeAll(). The first creates a new ExecutorService backed by a thread pool of the specified size. This is a generic ExecutorService that is designed to churn through tasks using all of the threads in the pool. invokeAll blocks until every task has finished executing. In this test, 10 threads will rip through 250 tasks, each making a single call to our service and capturing the result of that call. Each task execution results in a Future, which is a handle to the results of the task (and more).

Iterating over each resulting future, we make several assertions. The most important one is the last, where we assert that every task is given a unique Article. Thanks to the natural semantics of Set, this is easy to do in an elegant way. Another useful, though unexpected, feature is that if an exception occurs during the task execution, an ExecutionException will be thrown when get() is called on the corresponding Future. If our service fails for some reason, the test will fail because no exceptions are expected.

This technique makes simulating a multi-threaded environment in a test easy and readable. It’s important to only use this technique when it’s really necessary. The resulting test is more of an integration test than a unit test, and its run time is orders of magnitude more than a unit test, so overuse of the technique will artificially inflate the time it takes to runs the tests. After I’ve finished working on the component under test, I will reduce the test-data size and thread count to a level that the test still provides value, but is no longer a stress test (e.g. 10 articles and 2 threads). The next time the component is being worked on, the developer can crank up the values and run the tests to be confident that the behavior isn’t broken.

The complete source for a working example of this technique is available here. You’ll need Maven (or IntelliJ IDEA 7.x) to build and run the test. By default, the tests run against an in-memory H2Database instance, but if you look at you’ll see configurations for PostgreSQL and MySQL as well.

Happy testing!

Christian Nelson
Christian Nelson

Christian is a software developer, technical lead and agile coach. He's passionate about helping teams find creative ways to make work fun and productive. He's a partner at Carbon Five and serves as the Director of Engineering in the San Francisco office. When not slinging code or playing agile games, you can find him trekking in the Sierras and playing with his daughters.