mercredi 20 juin 2018

How can I say that a test (software) is correct?

I was having an technical interview with a big company that manufacture hardware/firmware.

For one of their product, they set up a team to test the firmware, because the firmware will not be upgradable (you don't want to ship 30 millions of bugged devices you know).

So their client gave them "specifications", the firmware team develop the firmware, and finally this team starts testing it with the help of C#, CANoe, and some dev boards.

So for example they flash the firmware into the board, then verify that the board(firmware) behaves like the specifications (for example it must return a specific message, or it must do X after Y, etc.... the usual unit tests stuffs).

These "unit tests" or "integration tests" are actually software themselves (written in C#), so there could be BUGS (for example a C memory leak) or "false positives".

For example this is a wrong test:

public bool TestEngineOffAfterTime(int timeToWait) {
  return true;
}

it's correct formally, compiles, but it's wrong.

The question I got was: HOW CAN YOU TELL THAT THE TEST IS CORRECT?

I left the interview, but didn't know the answer.

Can community help, pls? I can't find an answer by myself.

Aucun commentaire:

Enregistrer un commentaire