vendredi 31 août 2018

What would be the best way to run daily automated tests for a web scraper?

I maintain a REST API built with Django REST that, internally, scrapes several webpages to retrieve a bunch of information.

I have tests for every endpoint that check whether the scrapers are still working. They actually connect with the pages and check that the sources remain unchanged and that everything is still fine basically.

I would like to run these tests several times per day, and be notified when any of these scrapers fail. I'm not sure how should I approach this.

I've been looking at CI tools, but I'm not even sure if that's what I want.

I'm looking for something that allows me to:

1) Run tests automatically every X hours 2) Notify me of the results

What's the best tool for this?

Aucun commentaire:

Enregistrer un commentaire