Integration testing without Spring context

Should I always start Spring context during integration tests? (I mean using @SpringBootTest annotation)

Currently I'm writing integration test that involves a few classes and in order to make it faster I create object graph by hand (i.e. I don't start Spring IoC container). So currently my integration test (written in Spock) looks like this:

class UserConverterIT extends Specification {

    UserConverter converter = new UserConverter(new UserDtoFactory(new UserGroupPolicy()))

    def 'should ...'() {
        when:
        converter.convert(...)

        then:
        ...
    }
}

Alternatively I could add @SpringBootTest annotation, put @Autowire above UserConverter field and all dependecies would be injected automatically. However, the first approach is much faster. Is there something wrong in this approach?