Chris James

Developer and other things

Property-based testing in Go

Published on 07 March 2016

This posts hopes to illustrate

  • What property based testing is and why it is an important tool to compliment your existing tests
  • How easy it is in Go

Your current tests

The usual examples illustrating TDD could be described as example based tests; where you write tests by providing example inputs and expected outputs.

When you realise that they are just examples, you see your code perhaps isn’t as well-tested as you think.

Wouldn't it be nice if it were easy to provide thousands of different test inputs for your tests?

You may have already heard about property-based testing and think its about testing your code with randomly generated data to exercise different scenarios; but it actually requires a shift in mindset to get the most out of it.

Specifications, not just lots of data

Property-based testing is called that because in order to write the tests you have to think about the properties of your domain in order to write effective tests.

 An example with addition

We will test an add function, which just adds 2 integers together.

The tricky thing when you feed in random data, is how do you know what your expected result should be?

You may be tempted to do the above, but in effect you’re cheating because you’re using the implementation to verify itself.

Pretend that you’re testing a more complicated function. You know you shouldn’t do the following:

This isn’t really testing anything, if you change the implementation of complicated it will continue to pass its tests even if its behaviour is incorrect.

(This is a common problem with bad tests where the writer has re-implemented the unit they are testing, in the test!)

But what do you do? How can you verify your function on random inputs?

The properties of addition

What is there to addition anyway? Well, there’s a few things:

  • when you add zero to a number, you get the same number back again
  • the order of the inputs does not matter

Let’s write some property based tests based on these laws of addition.

The quick.Check function will run your assertion 100 times (by default), feeding in randomly generated data for the inputs required. If it fails at any point then you will be told for what inputs your function fails on so you can go reproduce it and fix.

Notice how:

  • By asserting on properties rather than known values we don’t have to "cheat" on the tests like I did earlier.
  • We have now described what addition is in terms of its properties, which is more expressive than the example based test from earlier.
  • We now have a more robust test suite, able to run thousands of different inputs in milliseconds.

Thinking abstractly about your domain

Once you take a step back and really think about the domain of your code, you may discover some unexpected properties that the examples you use in your general TDD flow wouldn’t find.

I imagine this is how a lot of QAs think. They tend to be a bit detached from the code we write and really think about the domain and how things can fall apart. Property based testing allows us to capture these kind of rules as tests.

Going beyond simple types

Property-based testing relies on generators for types to create random data for your tests. It’s simple to imagine how this works for integers but what about your own types?

All you have to do is implement the Generator interface for your type. Go will provide you with a rand to help you generate random data.

Conclusion

To write effective property-based tests you have to take a step back and think deeply about the rules of your domain.

Once you gain this deep understanding you can then express them very easily with property-based tests which will help you find bugs before you get to production.

The tooling in Go facilitates easily writing these specifications, even for your own types so there’s no excuse not to try it in your current project.

Notes