Skip to content


Unit testing async calls, multithreading, and rewriting delegates with expression trees

I am currently developing extensions for WCF. When you extend WCF, for example by writing a custom transport, you will find yourself writing a lot of APM-style methods (BeginXxx, EndXxx) in C#. Now, since I am also writing a lot of unit tests to test these async implementations, I run into the issue that multi-threaded tests are really not supported well in Visual Studio. The default Visual Studio test host will only handle exceptions/Assert.Xxx-calls on the main thread.

So when doing an Assert-Call on a method/delegate/lambda you pass as a callback to a BeginXxx-method, this code will execute as a callback on a different thread. If the Assert gets executed and fails (a failed call to Assert.Xxx raises an exception), the test will not report the failed assertion properly. An [ExpectedException(…)] attribute on your test will not work either. In the best case, the test will result in an error … other cases include the test host process crashing. The test won’t terminate properly though.

The best/most straightforward option you have to circumvent this is to catch any exception from a background/callback thread, and rethrow it on the main thread.

SqlCommand.BeginExecuteReader-sample

For example if you want to test the following code (just a sample! I am not going into discussions whether you should be testing against a sql database like this). We call SqlCommand.BeginExecuteReader, passing a callback-method as a lambda expression (the delegate is of type System.AsyncCallback). This screenshot contains only the logic I want to test (note: this uses an extension method public static SqlCommand CreateCommand(this SqlConnection conn, string commandText)). The SQL statement will wait for five seconds and then do a simple select on the sys.objects system view.

The highlighted part shows the lamdba expression for the callback logic we want to test.

image

This test code has a number of issues:

a) the code in the lambda will execute on a callback thread, none of the Assert.Xxx-statements will work properly.

b) the test won’t wait for the async callback to be completed, it would just exit the [TestMethod] TestAsyncSqlCommand before the callback actually executed. It would be handy if it could do an automatic blocking wait before the Assert.IsTrue(callbackExecuted) to wait for the callback to complete – which in this example would take at least five seconds (due to the waitfor delay N’00:00:05’ in the SQL query).

To make this test work, we would

a) need to enclose the callback code in the lamdba with a artificial try/catch-block, to catch any exception thrown by a failed Assert.Xxx call

b) add a artificial synchronization object (for example ManualResetEvent), which we declare at the top of the test method, then signal during the callback, and then perform a blocking wait before the Assert.IsTrue(callbackExecuted).

It would look like this … we have two threads in action, one is our main test thread which calls the BeginExecuteReader, then continues to the manualResetEvent.WaitOne-call and blocks there. The second thread will execute the callback method, everything in the lambda, and then signal the synchronization directive to unblock the main thread.

image

First of all, this is the same logic we want to test as in the sample before. It simply contains much more noise/clutter/infrastructure code we need just to make the test work. And for subsequent unit tests for other APM-implementations, we would need to replicate it over and over again.

What do we do, and how can we factor it out?

  • at the top of the test we declare the synchronization directive (ManualResetEvent) and a variable to store an exception
  • the lamdba expression is now wrapped in a try/catch/finally-block which a) collects the first exception thrown on the callback thread and b) signals the ManualResetEvent
  • does an explicit WaitOne (specifying a maximum timeout of 10.000 milliseconds) to wait for the callback to complete
  • if an exception has been collected on the background thread during the callback, we will rethrow it on the main thread. The test host will then report the failed Assert.Xxx properly

First things first: It is pretty easy to factor out the synchronization/WaitOne and the rethrow of the exception. We’ll (admittingly, I do have a serious IDisposable fetish) just create a class TestExceptionManager : IDisposable, add this class in a using() statement in the test and collect and rethrow the exceptions in the Dispose method. This class contains the synchronization directive (on which we will wait in the Dispose method), the exception-rethrow.

But we still need to inject code into the callback. We simply should not need to write the try/catch/finally for each callback method we use in our tests. This code should be automatically added by our testing infrastructure. Any delegate we pass as a callback needs to be automatically wrapped with a try/catch/finally block. And this solution needs to work for delegates of any type (not only for AsyncCallback, as in this sample). The parameter we pass as a callback method (lambda in our case) is always a delegate of a type determined by the BeginXxx/EndXxx method pair we are testing.

.NET 4.0 comes with the Dynamic Language Runtime and vastly enhanced support for expression trees. It is now possible to generate full method bodies using expression trees. We will use this to dynamically create our interception code. We will generate

  • a new method which has the same signature as the original delegate (mandated by the BeginXxx-method). For example, AsyncCallback is declared as delegate void AsyncCallback(IAsyncResult asyncResult)
  • a body containing a try/catch/finally block
  • in the try-block, invoke the “original” delegate (our lambda method from the first sample, with the correct parameters)
  • in the catch-block, swallow any exception and store it, so we can rethrow it later on the main thread (let’s call it void StoreException(Exception xcptn))
  • in the finally-block, signal the synchronization directive (let’s call it void DecrementCounter())
  • if the delegate type declares a return type other than void, return the correctly typed result (in case of a catch we may need to return the default value for this return type)

See this posting how I implemented a method which dynamically generates a new method body and delegate.

It is implemented in the

TDelegate TestExecptionManager.Wrap<TDelegate>(TDelegate del) where TDelegate : class

method.

The expression tree for the dynamically generated code will look similar to the following (for AsyncCallback).

.Lambda Fooo<System.AsyncCallback>(System.IAsyncResult $asyncResult) {
    .Try {
        .Block() {
            .Invoke .Constant<System.AsyncCallback>(System.AsyncCallback)($asyncResult) // this will invoke our “logic” for the callback
        }
    } .Catch (System.Exception $xcptn) {
        .Block() {
            .Call .Constant<VisualStudio.TestTools.UnitTesting.Extensions.TestExceptionManager>(VisualStudio.TestTools.UnitTesting.Extensions.TestExceptionManager).StoreException($xcptn)
            ;
            .Default(System.Void)
        }
    } .Finally {
        .Block() {
            .Call .Constant<VisualStudio.TestTools.UnitTesting.Extensions.TestExceptionManager>(VisualStudio.TestTools.UnitTesting.Extensions.TestExceptionManager).DecrementCounter()
        }
    }
}

Now we can write our test the following way:

image

Note that we just need to declare the TestExceptionManager in a using() statement. Aditionally, the tem.Wrap<AsyncCallback> call will generate a dynamic method with the same signature as the AsyncCallback-delegate type mandates, then create a delegate pointing to this method, and return the newly created delegate (which then gets passed as the callback to the BeginXxx-method). Most of the stuff is the logic we want to test, and much less infrastructure clutter.

Cheers.

You could also combine this with Chess from Microsoft Research. After installation, Chess will be available as a test host for MS unit tests.

Posted in .NET 4.0, Productivity, Testing, Utilities.

Tagged with , , , , .


0 Responses

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.



Some HTML is OK

or, reply to this post via trackback.