mercredi 26 février 2020

Should I unit test *what* or *how* for composite functions?

I have functions f and g, which already have a collection of unit tests ensuring that their behavior is correct for some known inputs + outputs pairs (plus exception handling, etc).

Now I'm creating the function h(), as follows:

def h(x):
    return f(x) + g(2*x)

What's a good approach for unit testing this function? Would this change if h() was substantially more complex?


My thoughts so far:

I see two possibilities for testing h().

1. Testing if h() is doing the correct "plumbing"

Mock out f() so that it returns y_0 when called with input x_0; mock g() so that it returns z_0 when called with input 2*x_0. Then check that calling h(x_0) returns y_0+z_0.

Advantages:

  • Very simple to implement the test.
  • Can quickly find bugs where I incorrectly connected the outputs of f and g, or where I called them with wrong arguments (say, calling g(x) instead of g(2*x) in h()).

Disadvantages:

  • This is testing how, not what h() should do. If later I want to refactor h(), then I'll probably need to rewrite these types of tests.
  • If the plumbing specified by the test does not produce the intended high-level behavior for h(), then these tests won't catch this error. For example, maybe the correct plumbing was supposed to be f(-x) + g(2*x), and I made it wrong both in the function definition and in the test definition.

1. Testing what h() should do

Let's say that the purpose of h() is to compute the sum of primes below the given argument. In this case, a natural test suite for h() would involve testing it with known input and output pairs. Something that makes sure that, for instance, h(1)=2, h(2)=5, h(5)=28, etc, not caring about how h() is computing these numbers.

Advantages:

  • This type of test checks that h() is indeed following its intended high-level behavior. Any plumbing mistakes that alter this will be caught.
  • Refactoring h() will probably not necessitate changing the test suite, and will even be easier, since the tests help us guarantee that the behavior of the function doesn't change.

Disadvantages:

  • In this simple example it's easy to produce such pairs because the mapping that h() performs is not very complicated (just sum the n first primes). However, for a very complicated h(), my only option for producing such pairs might be me to come up with an input x and compute the correct output by hand. This doesn't seem reasonable if h is very complicated.
  • Since coming up with known input-output pairs requires me to compute what f() and g() will produce given a certain input, there will probably be some duplication of effort, since I already spent some time doing that when creating the unit tests for these functions.

Related question: Unit testing composite functions.

This question is at first glance very similar to mine. However, the two most voted answers present completely different approaches to the problem (the two approaches I mentioned above). My question is an attempt in clarifying the pros/cons of each approach (and perhaps learn other approaches), and potentially establish which is best overall. If no approach is best in all cases, I would like to understand in which cases one should use each of them.

Aucun commentaire:

Enregistrer un commentaire