jeudi 21 février 2019

Python Dynamic Test Plan generation

I am using Sphinx for documentation and pytest for testing. I need to generate a test plan but I really don't want to generate it by hand.

It occurred to me that a neat solution would be to actually embed test metadata in the tests' themselves, within their respective docstrings. This metadata would include things like % complete, time remaining etc. I could then run through all of the tests (which would at this point include mostly placeholders) and generate a test plan from them. This would then guarantee that the test plan and the tests themselves would be in sync.

I was thinking of making either a pytest plugin or a sphinx plugin to handle this.

Using pytest, the closest hook I can see looks like pytest_collection_modifyitems which gets called after all of the tests are collected.

Alternatively, I was thinking of using Sphinx and perhaps copying/modifying the todolist plugin as it seems like the closest match to this idea. The output of this would be more useful as the output would slot nicely in to the existing Sphinx based docs I have though there is a lot going on in this plugin and I don't really have the time to invest in understanding it.

The docstrings could have something like this within it:

:plan_complete: 50 #% indicator of how complete this test is
:plan_remaining: 2 #the number of hours estimated to complete this test
:plan_focus: something #what is the test focused on testing

The idea is to then generate a simple markdown/rst or similar table based on the function's name, docstring and embedded plan info and use that as the test plan.

Does something like this already exist?

Aucun commentaire:

Enregistrer un commentaire