Automatically close xvfb after Selenium test using pyvirtualdisplay

I am running Selenium tests headlessly in Firefox through pyvirtualdisplay, and earlier encountered an unexpected error (connection refused) coupled with an extremely long response time between 75 and 80 seconds. After some experimenting, it turned out that my previous functional testing had left three instances of Xvfb running, and after manually killing these, my next test achieved an expected failure in only 5.7 seconds. So, problem solved, but ps reveals that this test, too, left an Xvfb process hanging around. Now, according to the docs, the stop() method seems like the best candidate for that, but including it in my code doesn't appear to have any effect. Here's the test in question:

from selenium import webdriver
from pyvirtualdisplay import Display
import unittest

display = Display(visible=0, size=(800,600))
display.start()

class NewVisitorTest(unittest.TestCase):
    def setUp(self):
        self.browser = webdriver.Firefox()

    def tearDown(self):
        self.browser.quit()

    def test_start_retrievable_list(self):
        self.browser.get('http://localhost:8000')    
        self.assertIn('To-Do', self.browser.title)
        self.fail('Finish the test!')

if __name__ == '__main__':
    unittest.main(warnings='ignore')

display.stop()

How can I alter the script to shut down the Xvfb instance without having to grab the PID and shut it down manually?