System.setProperty in junit tests

I have two tests, we'll call them test A and test B.

In test A we have this line

System.setProperty("<interface>", "<implementation>);

In test B we have that same line.

When both lines exist, test B fails. When I comment out the line in test A, test B succeeds. If I do a

System.clearProperty("<interface>") 

test B fails.

I have a similar issue with tests C and D with a different interface property. Test C sets it, test D does not. When I comment out the set call, D passes. When I don't. D fails. When I call clear, D fails.

Calling clear and commenting out the line both cause B and D to have the "correct" value in that system property However they fail when I call clear (despite having the same value as when the line is commented out) and succeed when commenting out. Anyone have any ideas why this is the case?