samedi 10 janvier 2015

Light-weight software development process acceptance test for helper utilities

I work at a place where we have a well-defined formal process from the point any idea is opened as an issue in the defect/feature tracking database till the point it goes out in a formal software release.


I personally enjoy process as much as the next guy since it results in higher-quality products with fewer defects, iron-clad and bullet-proof code, etc. etc. etc. Unfortunately over time, the process can get so rigid that it becomes difficult for smaller items to get through without tying the burden and overhead of the established process. This has now happened to us regarding our acceptance process.


For normal, ordinary executables and DLLs that contain service code and GUIs visible to the end user, this is fine for us:



  • Work Issues 3, 5, and 7 in the bug/feature tracking database for Gobbledygook.dll.

  • Update test documentation for Gobbledygook Package to exercise Issues 3, 5, and 7. This test documentation consists of manual test steps written at the keystroke even (e.g. Launch application, click File > Open, ....)

  • The Software Test team runs through the manual test procedures written in the test documentation, thereby exercising Issues 3, 5, and 7. If all goes well, Issue 3, Issue 5, and Issue 7 can be closed in the database because they have been accepted in Gobbledygook.dll version 2.3.4.5.


As I stated, we're fine with this process for Gobbledygook.dll.


Unfortunately, the same process applies to helper libraries whose sole purpose is to maintain reusable, shared code across the entire project. Suppose I have a Utilities.dll that contains a MultiMap.cs file, and Issue 8 was opened to fix a bug in MultiMap's Values property.


The theory is that Utilities.dll cannot live by itself; it provides no useful functionality to the end user; it's merely a repository of shared code to avoid copying/pasting the same MultiMap.cs file for example in each type of Gobbledygook deliverable. Since it does not stand-alone, it must be tied to something that stands alone in order for it to be accepted, and all Utilities issues going into a release of Utilities.dll must be accepted when going through the acceptance test process of whatever Gobbledygook is being released.


Going by the process, if I want to work on that bug fix to in Utilities.dll, I need to plan for it by finding some higher-level deliverable such as Gobbledygook.dll so that I can update its test procedure to exercise Issue 8 before I can close out Issue 8 for Utilities.dll.


If there is nothing high-level being worked in the current schedule, then I'm not supposed to work on Issue 8 until the need arises because I cannot close out the issue because there would be no audit trail from a manual test procedure to the Issue 8.


We're trying to get folks on-board with the idea that low-level Visual Studio automated unit tests should be sufficient to prove Issue 8 has been resolved in order to close it out.


The main concern regarding process is that if Issue 8 has not been tested thoroughly enough, then a bug might get out causing another change to Utilities.dll.


Our counterargument is that if a bug gets out of Utilities.dll, there's still the opportunity to catch it during Gobbledygook's testing. At that point, a rigid process will result in days of acceptance testing just for Utilities.dll before we can get back to testing Gobbledygook.dll.


On the other hand, a simpler, automated unit test-only acceptance test of Utilities.dll will allow the an accepted Utilities.dll containing the fix to get in the tester's hands within hours instead of days so that Gobbledygook's testing can resume.


Folks are coming on-board with these points; however, there's still the audit issue for following process.


How can I tie Utilities.dll's Issue 8 and automated unit tests to a formal test procedure in order to satisfy and close out Issue 8 for good without having to worry about scheduling issues for Gobbledygook.dll?


Aucun commentaire:

Enregistrer un commentaire