Requiring External Resources Before Attempting JUnit Tests

If you have an integration test that requires external resources to be available, like a local DynamoDB server, that test should be skipped rather than fail when the resources aren’t there. In JUnit, this can be accomplished by throwing an AssumptionViolatedException from an @BeforeClass method, or better yet, with reusable ClassRules.

A ClassRule runs like an @BeforeClass method; once before the entire suite of test methods in the class. @Rule is analogous to @Before and runs before each test method. In this case, we use the ClassRule to check for resources along with preparing the use of them (see the call to .client()). If the AWS client cannot connect to a local DynamoDB server, it will throw an AssumptionViolatedException causing JUnit to mark the test class as skipped rather than failed.

-------------------------------------------------------
 T E S T S
 -------------------------------------------------------
 Running DynamoDaoIT
 Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.054 sec - in DynamoDaoIT

 Results :

 Tests run: 1, Failures: 0, Errors: 0, Skipped: 1
import java.util.Properties;

import org.junit.Before;
import org.junit.ClassRule;
import org.junit.Test;

public class DynamoMappingsDaoIT {
  private static final Properties CONFIG = // load your app test properties;

  /**
   * This runs like an @BeforeClass method (@Rule is analogous to @Before).  If the AWS
   * client cannot connect to a local DynamoDB server, it will throw an
   * AssumptionViolatedException causing JUnit to mark the test class as <code>skipped</code>
   * rather than <code>failed</code>.
   */
  @ClassRule
  public static final DynamoDbRule DYNAMO_DB_RULE = new DynamoDbRule().withLocalPort(CONFIG.getString("dynamo.port"));
  
  private DynamoDAO dao;

  @Before
  public void setUp() {
    // Create this here to give the class rule a chance to run first.
    dao = new DynamoDAO(CONFIG);
  }

  @Test
  public void shouldUseDynamo() {
    // test safely knowing that your dynamo client is connected
    mappingDAO.find(1);
    assertNotNull(DYNAMO_DB_RULE.client());
  }
}
import com.amazonaws.AmazonClientException;
import com.amazonaws.ClientConfiguration;
import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient;
import com.google.common.base.Strings;
import org.junit.Assume;
import org.junit.rules.ExternalResource;

public class DynamoDbRule extends ExternalResource {
  private String endpoint = "http://localhost:8000";
  private String accessKeyId;
  private String secretKey;
  private int maxRetry = 0;
  private AmazonDynamoDBClient client;

  public DynamoDbRule() { }

  public DynamoDbRule withLocalPort(final int port) {
    return withEndpoint("http://localhost:" + port);
  }

  public DynamoDbRule withEndpoint(final String endpoint) {
    this.endpoint = endpoint;
    return this;
  }

  public DynamoDbRule withAccessKeyId(final String accessKeyId) {
    this.accessKeyId = accessKeyId;
    return this;
  }

  public DynamoDbRule withSecretKey(final String secretKey) {
    this.secretKey = secretKey;
    return this;
  }

  public DynamoDbRule withMaxRetry(final int maxRetry) {
    this.maxRetry = maxRetry;
    return this;
  }

  public AmazonDynamoDBClient client() {
    if (client == null) {
      throw new IllegalStateException("Run before() to create the client");
    }
    return client;
  }

  @Override
  protected void before() throws Throwable {
    ClientConfiguration clientConfig = new ClientConfiguration().withMaxErrorRetry(maxRetry);

    if (Strings.isNullOrEmpty(accessKeyId) || Strings.isNullOrEmpty(secretKey)) {
      client = new AmazonDynamoDBClient(clientConfig);
    } else {
      AWSCredentials creds = new BasicAWSCredentials(accessKeyId, secretKey);
      client = new AmazonDynamoDBClient(creds, clientConfig);
    }
    client.setEndpoint(endpoint);

    try {
      client.listTables(1);
    } catch (AmazonClientException e) {
      Assume.assumeNoException(e);
    }
  }
}
Advertisements

Integration Testing with DynamoDB Locally

One of the really nice things about using DynamoDB to back an application is the ability to write integration tests that have a good test server without trying to mimic DynamoDB yourself.

DynamoDB_Local is available from AWS and is easily incorporated into a Maven build. Take a look through the documentation for running DynamoDB on Your Computer for the parameters available.

The general steps for adding this to your build are:

  1. Download the DynamoDB Local artifact.
  2. Reserve a local port for Dynamo to start on.
  3. Start DynamoDB_Local just before integration tests are run.
  4. Use the failsafe plugin to run integration tests.

The steps are covered in more technical detail in the pom.xml file below. Adding this to the build is the only thing necessary as this performs all the steps above and tears down the process afterwards.

Unit Testing with HttpClient’s LocalTestServer

When unit testing code that uses HttpClient, it can get a bit tricky to not test against a active web server.  There are a couple of approaches to this to keep your tests at the unit level.

  1. Mock and inject HttpClient — While this is certainly possible and gives you complete control, it can take a lot of mocking and get tedious quite quickly.
  2. Use LocalTestServer from HttpClient — This is the point of this post and I will now explain.

Let’s take a look at the basic setup for getting this test server up.

Basic Setup

First, if you’re using Maven, you’ll need to bring in a new depedency.

  <dependency>
      <groupId>org.apache.httpcomponents</groupId>
      <artifactId>httpclient</artifactId>
      <version>4.0.1</version>
      <classifier>tests</classifier>
      <scope>test</scope>
  </dependency>

Next, you’ll need to setup your unit test with the test server.

public void setUp() {
    LocalTestServer server = new LocalTestServer(null, null);
    server.start();
}

That’s all it takes to get the server in place and started but it’s not very useful without some handlers.

Adding Handlers

To really get something worthwhile out of the server, you’ll want to register at least one handler.  Your handler must implement

org.apache.http.protocol.HttpRequestHandler

which has one really obvious method,

void handle(HttpRequest request, HttpResponse response, HttpContext context) throws HttpException, IOException;

Since we should be in a unit test, you can decide to either Mock this interface or create concrete classes that implement it.

Now that we have this spiffy way of controlling the response of the server, let’s see what it takes to register this handler with the server.

// do something to mock this or instantiate your own concrete class
HttpRequestHandler handler;

server.register("/someUrl/*", handler);

That’s all it takes to setup a local test server with at least 1 registered handler listening on /someUrl. For clarity, let’s take a look at the code all together.

public class MyUnitTest {
  private LocalTestServer server = null;

  @Mock 
  HttpRequestHandler handler;

  @Before
  public void setUp() {
    server = new LocalTestServer(null, null);
    server.register("/someUrl/*", handler);
    server.start();

    // report how to access the server
    String serverUrl = "http://" + server.getServiceHostName() + ":"
        + server.getServicePort();
    System.out.println("LocalTestServer available at " + serverUrl);
  }

  // do lots of testing!

  @After
  public void tearDown() {
    server.stop();
  }
}

EasyMock’s “N matches expected, M recorded” not always what you expect

This has been discussed in varying ways. I tend to point to this article as a good description of how to understand what is going on when this message is given. There is an interesting corner case that I feel needs to be explored as I’ve just wasted an hour trying to figure it out.

Consider the following code sample.

public void testSomethingBasic() {
  Thing myThing = new Thing(EasyMock.isA(Whatever.class));
  // do more stuff in the test
}
public void testSomethingElse() {
  IJobber myJobber = EasyMock.createMock(IJobber.class);
  EasyMock.expect(myJobber.doStuff()).andReturn(null);
  // do more stuff in the test
}

The thing to notice is that isA(..) was used on a non-mock object then used again on a mock object. The problem here is that EasyMock will record that you created a matcher (ie. isA(Whatever.class)) that was never used then will record that you created another one that you did use. You’ll get an error to the nature of “1 matcher expected, 2 recorded”.

While this is not always the case, a debugging method to consider is when you have more recorded than expected, check test methods that run before the one throwing this exception to make sure you aren’t improperly creating matchers that don’t get used.