mercredi 4 novembre 2015

Is it possible to dynamically run benchmark tests?

I have a few different implementations of an interface, and a variety of factors I want to test them under. End goal being to make a grid of results for different implementations in different situations.

I could write a test for each possible combination, but that gets exhausting:

func Benchmark_ImplA_N100_X300(b *testing.B){
   impl := newImplA(100,300)
   runBenchmark(b,impl)
}

The more combinations I add, the more I have to copy/paste. This gets cumbersome fast.

I would love to do something like:

tests := []testing.InternalBenchmark{}
for _, n := range []int{50,100,500,10000}{
   for _,x := range []int{300,200}{
      for name, gen := range implementations{
         impl = gen(n,x)
         bm := testing.InternalBenchmark{Name: fmt.Sprint(name,x,n)}
         bm.F = func(b *testing.B){
            runBench(b,impl)
         }
         tests = append(tests,bm)
      }
   }
}
testing.RunBenchmarks(anything, tests)

so that I can update the list(s) of combinations, and benchmarks will magically run in all combinations. I have tries something like this in main and in TestMain, and nothing ever outputs. I am not sure if I am using it wrong, or if the testing package is just doing something funny.

I really don't care if the go test tool can handle it or if there is some other way.

Aucun commentaire:

Enregistrer un commentaire