24 Days of Hackage: doctest

Testing and documentation. Two words that will make even the most hardened programmers shudder. Unfortunately, they are ultimately two of the most important aspects of programming - especially if you want your work to succeed in the wild. Even if you practice test-driven development religiously, you still can’t rule out writing documentation. If only there was a method to combine the two…

Simon Hengel’s doctest library is one solution that can ease this pain. Modelled off the doctest library for Python, doctest embeds tests inside the documentation of modules. The idea is: if testing requires code to be tested and an expected result, then we can treat this as an expected interaction at a REPL. So the work required by the programmer is to enter the input and output of a REPL session. doctest then parses the documentation and runs the code, checking that your expectation matches reality. If so, then it naturally follows that your documentation is consistent with what the library does.

doctest for Haskell works using Haddock’s documentation strings. For example, we can easily check that a square function does indeed square its input:

{-| Given an integer, 'square' returns the same number squared:

>>> square 5
25
-}
square :: Int -> Int
square x = x * x

To run this, we have two options. One is to simply use the doctest executable:

However, in bigger projects you’ll want to integrate doctests into your Cabal file. This means that running cabal test will also run doctests, which makes it harder to forget to run them. To do this, we just have to build a little executable that calls the doctest function:

main :: IO ()
main = doctest [ "2013-12-17-square.hs" ]

The output of this is the same, but we can now easily integrate this as a test-suite in a Cabal file.

So far we’ve only see a simple test of a pure function, but doctest can go a lot further. For example, we might have a function that requires a callback. In a GHCI session, we might write this callback in a let binding, and we can do the same in doctest. The function we are testing will work in the IO monad, and the expected behaviour is to print to standard output. This is naturally expressed in doctest - we just write out what we’d expect to see in GHCI:

{-|
>>> :{
      let callback name = do
            putStrLn $ "Hello. Yes, this is " ++ name
>>> :}

>>> printer "Dog" callback
Dog says:
Hello. Yes, this is Dog
-}
printer :: String -> (String -> IO ()) -> IO ()
printer name callBack = do
  putStrLn $ name ++ " says:"

Observant readers will not that this isn’t quite what it claims - and doctest notices that too:

Whoops, looks like we forgot to actually call the callback! If we fix that, we get a happy result from doctest once again.

If you’re a library author, I highly recommend you give doctest ago - it’s usage really is a net win. You get more guarantees that your library is doing what is expected of it, and you get even more back if you encourage your users to help write documentation. By writing examples for you, they’ll also be writing test cases - and probably test cases for the things they care about too.