samedi 16 mai 2020

How can you track internal variables & verify path coverage during white box testing?

Reading around whitebox testing, I understand it is an approach that focuses more on testing the implementation (rather than the interface), and the various paths that can be internally taken. I can see how it can uncover bugs otherwise undiscovered when black box testing.

However, I'm yet unable to find a practical complete example (not pseudocode) that does this. This website (https://www.guru99.com/white-box-testing.html) suggests CppUnit is a 'white box testing tool'. Looking at the CppUnit documentation (link), it looks like a useful library, however was unable to find any examples that explicitly focused on statement/path coverage, or tracking internal variables, from a quick look.

Is anyone able to share approaches or industry conventions for white box testing that verifies the actual paths taken by the code, and verifies that all the internal variables are modified at the right points in the code, and to the values we predict? Are you able to share any practical examples?

To illustrate my question, consider a crude simple function we want to test with the 'white box' approach. Function shouldStreetLightBeOn() determines whether the street light should be on based on ambient light level.

bool shouldStreetLightBeOn (int ambientLight_lux)
{
  bool result = true;   //default to turning street light on

  if (ambientLight_lux < 100) // if current light level < 100lux
  {
     result = true; //street light should be on
  }  

  else 
  {
    result = false; //street light should be off
  }

  return result;
}

From a 'static inspection', I predict there are 2 paths through the function; one where ambientLight_lux is less than 100, and another where its 100 or more. Thus, I devise two test cases-

TEST_CHECK(shouldStreetLightBeOn(100) == false);  //day
TEST_CHECK(shouldStreetLightBeOn(99) == true;)  //night

However, how do I know whether the paths I predicted the function would take based on the inputs, are the actual paths taken by the function? How do I know the values were modified at the right points of code I predicted? I imagine I can manually step through the function with a debugger, while 'Watching' all the internal variables, and manually verify my predictions. However how can I automate this?

To know which path was taken, perhaps one way is to modify the production code slightly to change a variable accessible to the test case when the different paths are reached, using #ifdef & #endif directives.

bool shouldStreetLightBeOn (int ambientLight_lux)
{
  bool result = true;   //default to turning street light on

  if (ambientLight_lux < 100) // if current light level < 100lux
  {
     result = true; //street light should be on
     #ifdef DEBUG
     //set a variable that is accessible to the test case
     #endif
  }  

  else 
  {
    result = false; //street light should be off
    #ifdef DEBUG
    //set a variable that is accessible to the test case
    #endif
  }

  return result;
}

However, perhaps this can get unmanageable? Also, by having potentially global variables, this may infringe upon the 'hide the implementation' principle the production code should have in-service?

On the other hand, if I maintain a parallel copy of my production code fully instrumented for the test case, I imagine I'll quickly run into maintainability problems when needing to change the production code. Not to mention the lack of confidence from not directly testing the production code.

Looking forward to thoughts & ideas.

Aucun commentaire:

Enregistrer un commentaire