Python mocking not working for shutil.rmtree

I have a class with a method which uses shutil.rmtree to remove some files if a param is passed as true, How to mock this behavior so as other tests don't break which needs these files.

My class looks like this -

class FileConverter(object):

    def __init__(self, path_to_files):
        self._path_to_files = path_to_files

    def convert_files(self, rmv_src=False):
        doStuff()
        if rmv_src:
            shutil.rmtree(self.__path_to_files)

    def doStuff():
        # does some stuff

Now my tests look like -

class TestFileConverter(unittest.TestCase):

    def test_convert_success(self):
        input_dir = 'resources/files'
        file_converter = FileConverter(input_dir)
        file_converter.convert_files()

        # assert the things from doStuff

    @mock.patch('shutil.rmtree')
    def test_convert_with_rmv(self, rm_mock):
        input_dir = 'resources/files'
        file_converter = FileConverter(input_dir)
        file_converter.convert_files(True)

        self.assertEquals(rm_mock, [call(input_dir)])

Now when I run this testsuite the test with rmv gives me assertionError

<MagicMock name='rmtree' id='139676276822984'> != [call('resources/images')]

and the first test gives me file not found error since the mock did not work and the rmv source test removed the file

FileNotFoundError: [Errno 2] No such file or directory: 'resources/images'

If I comment out the second test with rmv_source true then my first test works fine. What am I doing wrong here?