dimanche 13 septembre 2020

How to create dynamic testset in Julia?

Using Julia's Test package, how can I create @testsets that reference a value at runtime?

E.g. I have

test_data = [
   (model="a",datapath="models/a.json"),
   (model="b",datapath="models/b.json"),
   ...
] 

And I would like to do this:

@testset "models" begin
    for test in test_data
        @testset test.model begin
            
            # test range of values/functions here
        end
    end

end

But I get an error like this, which I assume is because I'm trying to define the testset name at runtime.

ERROR: LoadError: LoadError: LoadError: Unexpected argument test.model to @testset
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] parse_testset_args(::Tuple{Expr}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Test\src\Test.jl:1244
 [3] testset_beginend(::Tuple{Expr,Expr}, ::Expr, ::LineNumberNode) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Test\src\Test.jl:1087
 [4] @testset(::LineNumberNode, ::Module, ::Vararg{Any,N} where N) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Test\src\Test.jl:1079
 [5] include(::String) at .\client.jl:457
 [6] top-level scope at C:\MyPackage\runtests.jl:7
 [7] include(::String) at .\client.jl:457
 [8] top-level scope at none:6

Because I'm running hundereds of tests in each inner @testset it would be really nice to have the results grouped when running the tests. Also, when I get an error it tends to be in many results and my terminal loses the history above where the error starts, so printing the model name before each set of tests doesn't work very nicely either.

Aucun commentaire:

Enregistrer un commentaire