How can I dynamically generate test cases with common test?

With Common Test test suites, it looks like test cases must be 1:1 with atoms that correspond to top-level functions in the suite. Is this true?

In that case, how can I dynamically generate test cases?

In particular, I want to read a directory, and then, (in parallel) for each file in the directory, do stuff with the file and then compare against a snapshot.

I got the parallelization I wanted with rpc:pmap, but what I don't like is that the entire test case fails on the first bad assert. I want to see what happens with all the files, every time. Is there a way to do this?