dimanche 21 juillet 2019

Testing Spark: how to create a clean environment for each test

When testing my Apache Spark application, I want to do some integration tests. For that reason I create a local spark appliciation (with hive support enabled), in which the tests are executed.

How can I achieve that after each test, the derby metastore is cleared, so that the next test has a clean environment again.

What I don't want to do is restarting the spark application after each test.

Are there any best practices to achieve what I want?

Aucun commentaire:

Enregistrer un commentaire