Which one is better? To test by data or test by algorithm?

We are developing a Play/Scala application and using Specs2 for our tests. Most of our tests are for controllers.

We've created a data sample generator for each entity of the database and use them to generate the data for tests. These sample generators try to generate various cases of data scenarios.

For each test we:

• Insert the data using the sample data generators
• Call the controller action under test
• Calculate the correct answer
• Assert if returned result is equal to the calculated one.

To make sure that we don't repeat a bug in both tests and main code, we always implement the test calculation in different way.

For example if there is a database query inside the system under test we fetch all data and calculate the result in memory for the corresponding test.

I think our approach is somehow like property-based testing. Example:

"a * a returns the correct value" >> {
  forAll { (a: Int, b: Int) =>
    a * a === (1 to b).fold(0) { (acc, b) => acc + a }
  }
}

Another approach would be to generate specific data for specific cases and expect a fixed result based on the generated data which might be calculated by hand (not by code). Like this:

"2 * 2 == 4" >> {
  2 * 2 === 4
}

"3 * 5 == 15" >> {
  3 * 5 === 15
}

// etc.

NOTE: Our system is much more complicated than this simple scenario

The problem with our approach is that when our tests fail it's hard to find out if it's the test or main code causing the failure!

On the other hand, we have to write much more tests to cover all data scenarios for the other approach and it's hard to be DRY. As you can see in the above test examples.

In short, we have two options (of which we aware):

• Smart and complex but less number of tests
• Stupid and simple but more number of tests

Which one is recommended for real world applications and why?