node http server stop doesn't shut down connections

I've been looking for a while in a lot of threads about how to shut down an http server in nodejs.
I couldn't find a simple way to test my web app.

I'm trying to test my server, writing integration test.
I want to simulate errors as well, hence I start the server with different configurations.

For example:

const app = express()
...
app.use('/route', createRouter(argA, argB)
const server = http.createServer(app)

I want to test that if argA throws an error, my server return 500,
as well as argB.

So I want to start the server with a mock of argA that throws,
then close the server and start it again with a mock of argB that throws.

But, the http server of nodejs doesn't close connected sockets,
so I cant do something like this:

beforeEach((done) => server.start(done))
afterEach((done) => server.stop(done))

Where start execute listen and stop execute close.
The server fails to start since the port is already in use.

How can I manage to start and stop the server with different configurations?