samedi 15 septembre 2018

Open source software with a set of well-constructed inputs for coverage measurement

I am evaluating my coverage tool (with respect to line coverage and statement coverage, etc.), so just want to know whether there exists an open source software, written in C/C++, with a decent amount of (better over one hundred) inputs for functionality testing. Then I can run the provided test cases towards the software to test my coverage tool.

While I can use a fuzzing tool (e.g., AFL) to start to from a seed file and quickly generate tons of inputs, my preliminary testing shows that the generated inputs cannot achieve a high program line coverage. For instance when fuzzing bzip2 the line coverage is always somehow around 40%, and they cover very similar paths. The generated inputs are not "diverse" enough.

Again, since I am evaluating a coverage measurement tool, I would like to find a open source C/C++ program with a large amount of (diverse) test inputs. That would be superb.

Also, for some well organized performance suite, such as SPEC 2006, they only provide three inputs for each benchmark program, which seems quite insufficient.

Could anyone send some pointers on this? Thanks.

Aucun commentaire:

Enregistrer un commentaire