dpdk Open Source

In an earlier post I wrote something about unittesting with the asunit framework.

In this post I want to shed some light on asynchronous testing and a minor modification we made to the asunit package to be able to run asynchronous tests that can be used to do some flow testing, especially important when the order of the calls to a remote server is important.

Asynchronous testing is done on code that does not return an immediate result. This is the case when a method is called, which communicates with a service (for example via flash remoting or with a flash media server). As a call to a remote service is non-blocking in flash (allowing the main program flow to continue), the result will be handled by an event and the event will be fired if or when the call has finished and we are getting some result back into the flash player.

In our case, we want to be able to use and test our systems even if they are based on remote communications based on some dataservice which may or may not be available at the time of code production. This can be achieved in multiple ways, but two important ideas are 1) using a stub and 2) asynchronous testing.

While developing a piece of software that depends on a remote service that has not yet been defined, we are able to continue development by means of injecting a stub in our application (a form of dependency injection). The stub stands in for the real code and has the same interface, so the application does not know that the stub is not the real service, but gets back some result that it is able to use in the program flow that is delivered by the stub in any way the stub likes. Most of the time this will be some form of dummy data that is usable as a test. A stub setup could look like this:

//the stub is injected in the dataservice in the application, possibly via a configuration setting
public static function main():void{ 
//dataService takes an argument of an FMSService, the stub could easily be a subclass, or both the stub and the service itself can have a superclass/interface
var ds: DataService = new DataService (new FlashMediaServiceStub(gateway));
var application = new Application();
application.setDataService(ds);
ds.addEventListener(RemoteIdEvent.NEW, onRemoteId);
ds.getRemoteId();
}
 
//method defined on the stub
public function getRemoteId():void{
//normally we would call a service which gets back an asynchronous result and then fires an event.
//now, we only fire an event with a random id
dispatchEvent(new RemoteIdEvent(RemoteIdEvent.NEW, new RemoteId(Math.ceil(Math.random() * 100))));
}

The ability to succesfully implement stubs is highly dependent on the architecture of your system, but is alway possible to refactor to a useful pattern

Asynchronous testing is used when the remote service is in place. The flow of a typical setup could look like this:

1. setup your session
2. login/register
3. get some information from the server
4. logout

As you can see, order is important here.

It becomes clear now that what we need is a way to stop code execution in our test while performing an asynchronous call and a way to be able to make sure our test methods get executed in the order we would like. Asunit provides the former, but not the latter.

Asynchronous testing is baked into the asunit framework and some info can be found on the blog of the developers of asunit right here. The functionality of asynchronous testing revolves around the

addAsync(method, timeout)

method in the TestCase class. when addAsync is used in a testMethod, all further execution of testMethods that are defined in the TestCase is delayed until the method is executed or until the timer is exceeded. Only one addAsync can be used per testMethod and it cannot be used in the constructor.
an example from a test with flash remoting in our package:

//we want our onBoolean result handler to be called within 2000 milliseconds, else our test fails
//stop execution of other tests untill we get a result
instance.addHandler("testBoolean", addAsync(onBoolean, 2000), onBooleanStatus);
			instance.testBoolean(true);

To tackle the problem of order, we need to be able to make our TestCase run in a predictable order. The easiest way to achieve this is to sort the testmethods (all public methods in a TestCase that are prefixed with “test”) by alphabet, that is, testA() is performed before testB().

Before we give the solution, let me state that proper unittesting assumes that the fixture (the environment in which the test takes place, all variables that are available, all data necessary to do a test etc.) is only created once (during setupO, and that the order in which testmethods are called should not be relevant in further code testing. While this is theoretically sound, it is pragmatically unsound. We just want to be able to run and test a certain path through our application. The theoretical way out of this, is to make different testcases with different fixtures and to just test the call we want to test, but this could lead to code that is harder to maintain (class explosion in our test code) and difficulties in providing the fixture.

We need a way to tackle this. In our case, the solution was easy:
this simple line was added to the TestCase class in the asunit framework in the setTestMethods method.

testMethods.sort();

What is even better, is keeping the framework as it is, so that asynchronous tests run in a non predicatable order (or the order specified in the setTestMethods method). Following the open/closed principle, we can just override the setTestMethods call since every test class extends TestCase:

override protected function setTestMethods(methodNodes:XMLList):void{
super.setTestMethods();
testMethods.sort();
}

In this way, we do not have to alter the framework (closed) but we can alter specific behaviour for some concrete cases (open) outside of the framework.

What would be even better than that is that the framework would implement a slight refactoring where the sorting of the testMethods is explicitely stored in a subroutine in testCase (for example: sortTests()) that does nothing by default, but can be overridden in specific cases such as ours. This would make the intention more explicit by providing a seperate method and relieves us from knowing things that are not interesting for a subclass of TestCase such as a method signature that takes an XMLList as a parameter.

Anyway, it is just another great example of the use of opensource frameworks: if you need a feature, just add it