Apply a test rule for a specific test only

I am using the READ_CONTACTS permission in my app and I want to test it in espresso. Currently, I am applying the rule:

@get:Rule var readContactsPermission = GrantPermissionRule.grant(READ_CONTACTS)

in my tests but this grants the permission to all methods. Is it possible to just apply the rule to specific tests or cancel the rule in other tests?