dimanche 26 juillet 2015

Tools for combining testing with monitoring?

Forgive me if this is in the wrong place. I'm asking the question in part because I don't know where this conversation belongs.

I think testing is great but I think the time has come to merge testing with monitoring. Today I use TDD to define tests, write code to pass them, then watch a continuous deployment service deploy the application to the web/cloud. The problem with this model is that the tests are run only once but the increasingly complex application has a life of its own. Why not run the tests continuously on the application so I have constant assurance that it's working as specified?

I'm inspired/influenced by Steve Yegge's famous rant about what he learned at Amazon:

monitoring and QA are the same thing. You'd never think so until you try doing a big SOA. But when your service says "oh yes, I'm fine", it may well be the case that the only thing still functioning in the server is the little component that knows how to say "I'm fine, roger roger, over and out" in a cheery droid voice. In order to tell whether the service is actually responding, you have to make individual calls. The problem continues recursively until your monitoring is doing comprehensive semantics checking of your entire range of services and data, at which point it's indistinguishable from automated QA. So they're a continuum.

Are there any tools for supporting this? I think I want to stop using, e.g., just Mocha/Tape for my Node.js tests and instead use Mocha as a monitoring plugin for Sensu (or some other popular monitoring tool). I want to combine traditional monitoring of health checks and business KPI metrics with periodic tests of the application in production that check 100% of the code every few hours, forever.

Are there tools/ideas/key words/conventions to support this exploration? Am I missing something obvious?

Aucun commentaire:

Enregistrer un commentaire