How to release a resource from class.getResource()

In a test I am trying to test for the correct behaviour when a configuration file is absent.

    given:
    File settingsGradleFile = new File( 'src/main/resources/settings.gradle')
    settingsGradleFile.renameTo 'src/main/resources/settings.gradle_OLD'

    ...

    when: 
    App.instance.start(mockStage)

    // now rename it back to its original name
    // NB in practice I put renameTo() in the "cleanup" block - but I have tested in both
    boolean result = settingsGradleFile.renameTo 'src/main/resources/settings.gradle'
    log.debug("result of rename: $result")

The first renameTo works fine. The second returns false (and the file name stays at "settings.gradle_OLD"). In the code of the CUT we have:

        URL settingsGradleURL = App.class.getResource( '/settings.gradle' )
        ConfigObject conf = new ConfigSlurper().parse( settingsGradleURL )
        projectName = conf.rootProject.name

I may be wrong, but it seems that the second renameTo is failing because in the mean time this getResource() operation has happened. How can I get this resource to be "released"?