How to test API client using mocks really works

I made an API client (a custom Django Auth Backend) that integrates with a company authentication API.

I used Responses to mock out the API as this offered many benefits, fast tests, works with our CI, isolated etc etc, and the result has been great so far.

The real question though is does my client really work?

I was thinking of spinning up a test instance of the API mentioned and then running my tests against that. It would be more painful as I would have to write scripts to restore the db to a known state etc. and it probably wouldn't work with our CI (Codeship), but at least I would have confidence things work as intended.

Then I would have to somehow toggle on and off Responses to be able to do this.

What are some simple ways of solving this problem?