How to implement integration testing for IdentityServer4?

This is more of a general question not a specific one, but I hope it might help someone in the future with the similar problem.

I am implementing IdentityServer4 for my business case where I initially want to set up authentication and authorization for a public API access. For that I will be initially using only ClientCredentials flow. For manually testing the usage flow and correct configuration and setup, I have created a small test API and a Client console app as in the sample projects from the documentation. But I want to automate it as I have almost completed the IdentityServer4 setup and am going to start working on the actual API itself.

So basically I want to do some integration testing against my implementation of the IdentityServer. But I have minimal experience regarding this topic so maybe you would have some suggestions or tips on how to set it up.

Should I set it up on the API side, when at least some endpoints are already done? But if my IdentityServer and the API are in different solutions, then for example for running these tests locally, I would still have to start IdentityServer manually and then could run the tests on the API project which connect against it?

Or for me it would make more sense to have the tests, the test api and client in the IdentityServer project (much in the same way it is done in the source code solution). But when looking at the source code integration tests, I saw they had a separate Startup class for IdentityServer startup. Should I do this similarly or rather configure the tests to run against the Startup class of my own IdentityServer project? Then I would have to copy all the appSettings files etc.

I know it's quite a general question and topic, but any pointers will be appreciated on how to actually test the correct thing. I know I don't need to test a lot about the functionality itself as it's also done in the source code solution, but I would rather have the integration tests for testing my own configuration etc, so if any of the components fail to communicate to each other, I would know immediately.

Thanks in advance!