Introduction to convention tests in .net

In one of our project we are using RabbitMQ queue for communication. In C# there is a popular library called MassTransit that (besides other things) allows you to use RabbitMQ. It is non trivial to test communication through external queue. At the same time there are many things that you need to remember in order to see message sent by a publisher later received by a consumer. That makes testing even more difficult, so we tried to unit-test as much things as possible to minimize scope of manual testing. One of the many things that you need to remember is that event class should have correct serialization attributes on each of property that we want to send. So we wrote the test that takes all implementations of our marker interface and for each of them we’ve checked if proper attribute is set on properties.

Test looks more or less like this:

typeof(IMassTransitEvent).Assembly.GetTypes()

    .Where(t => typeof(IMassTransitEvent).IsAssignableFrom(t))

    .SelectMany(t => t.GetProperties())

    .Where(PropertiesWithoutSeralizationAttribute())

    .ToList()

    .ForEach(FailI());

Because this code scan whole Assembly – with one test we’ve covered all existing event classes and all future ones. All of that thanks to power of reflection in C#. This test reminds me very good talk that I’ve seen some time ago where Maciej Aniserowicz was talking about so called convention tests. The talk you can find here: dotNetConfPL 2015: Testy jednostkowe w praktyce vol 3: testy konwencji (Maciej Aniserowicz)) . I recommend you to watch it if you can understand Polish language. I watched this talk again and I decided to refresh my knowledge in this area and make some more research by myself.

Let’s start with quick attempt to define what convention test are. In my opinion, convention tests are the tests that does not check any business logic. Instead of that they test for some rules that either:

  • team decided to introduce (for example naming convention, or that specific types of classes should exists in some exact namespace, etc.), or
  • rules that we are forced to follow because of some technical reason (eg. serialization attributes for mass transit messages) and there are no other easy options (like compilation failures) to check

From Maciek’s talk we can find that there are three main ways to write such tests:

  • using reflection to dig inside assembly
  • dig inside other project files
  • use roslyn to dig inside syntax tree

You might feel that it is better to use some tools for static code analysis. In some cases you may be right – but if you can do it with unit test framework – you don’t need any additional tools. You CI infrastructure will do all the job. Besides that you are using powerful part of c# language which is reflection so you can came up with test that are not possible to cover with other tools. So in my opinion it is useful to at least take a look what you can do and decide if it is worth to adopt in your project. In this case reflection is like very small building blocks – it is sometimes hard to imagine what you can build, but in fact you can build anything you want. So one of my goals for the next articles is to show you various mechanisms for convention tests.

I don’t want to tell you that you have to use them. In fact I think that many of you will never use any of those techniques, and it is OK. I came up with a simple rule of thumb when you would like to try convention tests. Imagine the situation – you’ve finished writing some part of code. You covered all business cases in unit tests and you took care about high quality of your code. Everything is compiling and all tests are green on CI. You deployed and now you want to manually test couples of scenarios… And it does not work at all. You are confused but after couple minutes of investigation you have this thought: “Oh, yes – I supposed to do [ some technical action that nothing forced you to do at first place ]”. This is the good moment to think about convention tests.

In next couple of articles I would like to explore this topic further.

Code for the whole series is available here: https://github.com/mprzybylak/CSharpConventionTests