Test-Driven Development for Scientific Programming, Part 2: Getting Started with Python

# More Realistic Testing Scenarios

In the last part I outlined a brief description of TDD using the example of writing a function in Python that adds numpy arrays together. Of course in actual research, the problem you are trying to solve with code is usually a lot more complicated than just adding a few arrays. Usually you have a more complex calculation consisting of many steps, which might be performed on data in a processing pipeline. You might be attempting to design an analysis routine for some processed data. You might be trying to recreate a lengthy theoretical calculation. In any of these scenarios, testing will be more difficult.

To illustrate a more realistic case, let’s imagine we wanted to design a class that models some data with a linear fit plus random noise, of the form $$y = \mathrm{slope} \times x + \text{additive constant} + \mathrm{noise}$$

$y = \mathrm{slope} \times x + \text{additive constant} + \mathrm{noise}$

We’ll call it LinearModel (by the way, it is convention in Python to use snake_case for functions and CamelCase for classes). This is in the context of wanting to integrate this class into a larger program.

In the last part we wrote tests that looked like this structurally:

def test_a_thing():
blah_blah_blah

Then we just ran these tests by calling them in the same python file. This is not a particularly good way to run tests, because you have to hard code every test failure message, and you will frequently have no idea why particular assertion failed. I suggest you use a tool like pytest, which will autorun functions with the name “test” in them, and give you an output like this:

======================= test session starts ======================
platform darwin -- Python 3.7.3, pytest-4.6.2, py-1.8.0
rootdir: /Users/dylan/Development/intro-tdd
collected 1 item

test.py F                                                    [100%]

============================ FAILURES =============================
__________________________ test_a_thing ___________________________

def test_a_thing():
>       blah_blah_blah
E       NameError: name 'blah_blah_blah' is not defined

test.py:2: NameError
==================== 1 failed in 0.06 seconds =====================

shell returned 1


It will also print out the values of any variables involved in a failing assertion.

In addition to pytest, I have begun to use an extension for pytest called pytest-describe, which changes how pytest looks for test functions. Let me write a quick test for our LinearModel class using pytest-describe-type test functions:

import numpy as np

def describe_linear_model():

def describe_y():

def when_noise_is_zero_it_returns_a_line():
linear_model = LinearModel()
linear_model.slope = 1
linear_model.noise_level = 0

xs = np.linspace(0, 2, 10)

ys = linear_model.y(xs)

assert np.all(ys == xs)

Here we’ve nested the test function under two “describe” functions. Notice that we also don’t need to have the word “test” in our functions for pytest to work. All that pytest-describe is doing here is to allow pytest to recurse through these nested describe-functions, find the actual test functions, and run them.

So what does this actually output?

======================== test session starts ========================
platform darwin -- Python 3.7.3, pytest-4.6.2, py-1.8.0, pluggy-0.12.0
rootdir: /Users/dylan/Development/intro-tdd
plugins: describe-0.12.0
collected 1 item

test.py F                                                      [100%]

============================= FAILURES ==============================
describe_linear_model.describe_y.when_noise_is_zero_it_returns_a_line

def when_noise_is_zero_it_returns_a_line():
linear_model = LinearModel()
linear_model.slope = 1
linear_model.noise_level = 0

xs = np.linspace(0, 2, 10)

>       ys = linear_model.y(xs)
E       AttributeError: 'LinearModel' object has no attribute 'y'

test.py:19: AttributeError
===================== 1 failed in 0.13 seconds ======================

shell returned 1


This seems like a purely aesthetic change, and it is, but I find that writing tests this way undoes a certain kind of test-writer’s block that can hit when trying to think of how to write tests for a new feature in a system. When writing a test like the one above, my workflow is something like:

• Imagine a central object to handle the job I need done (e.g. the LinearModel class)
• Start a test function describing this object (e.g. describe_linear_model, or for methods nest deeper e.g. describe_y)
• Write down a core behavior or property I think my object must have in order to function correctly
• I use the name of the test function to quickly express this in compact plain-English, e.g. when_noise_is_zero_it_returns_a_line

This works well because it helps me quickly work through the process of isolating core properties the object or function needs to have, and then gives me a nearly plain-English description of these with a working example.

This workflow took a fair bit of time to figure out, and will undoubtedly change as I continue to refine it. Being productive with TDD doesn’t happen immediately but requires an adjustment period of getting used to approaching a coding problem differently. There are many other testing tools available for Python alone, so if this kind of workflow isn’t for you, don’t be afraid to explore and experiment with other styles of TDD.