jeudi 9 janvier 2020

Mixpanel events test automation - any successful examples?

I’m trying to solve the puzzle with Mixpanel events test automation (mobile app). There’s a ton of events and parameters firing per different user scenarios in the app and regression testing of these takes a lot of manual QA time: they have to trigger all the scenarios manually.

What I’d like to verify is:

  1. the events are fired in the right place/time with correct targeting parameters.
  2. the information is reflected correctly in the Mixpanel DB without garbage (strictly required by our experimental design team).

What have we tried so far:

  1. Catching the data via proxy. (cons: events are batched and sent in groups, it’s difficult to match those with executed tests).
  2. Setting up an identical analytics test server, where we’re sending the duplicates of the calls. Downloading raw data from DB and using search algorithms (at least some results there, but we don’t have information on how Mixpanel transforms the data before storing it inside own DB. So this appeared to be overcomplicated and still not providing consistent results).

Why won’t we do that manually? Well, we have covered 81% of the Full regression test suite with Appium-based tests, and manual QA has to repeat scenarios just for analytics testing. There should be some way to integrate analytics testing into our automated suite.

Any ideas approaching this differently?

Aucun commentaire:

Enregistrer un commentaire