Race condition in httptestserver when changing handler

I have a slice of tests and I want to run them over one instance of httptest.Server. Each test has its own handler function.

func TestAPICaller_RunApiMethod(t *testing.T) {

    server := httptest.NewServer(http.HandlerFunc(nil))
    defer server.Close()

    for _, test := range testData {     
        server.Config.Handler = http.HandlerFunc(test.handler)

        t.Run(test.Name, func(t *testing.T) {
           ... some code which calls server
        }
    })
}

This code gives a race when running with "go test -race". It is probably because server runs in goroutine and I'm trying to change a handler concurently. Am I correct ?

If I try alternative code where I create new server for every test, then no races:

func TestAPICaller_RunApiMethod(t *testing.T) {

    for _, test := range testData {     
        server := httptest.NewServer(http.HandlerFunc(test.handler))

        t.Run(test.Name, func(t *testing.T) {
           ... some code which calls server
        }

        server.Close()
    })
}

So first question what is the best way to use one server for a slice of tests and chane handler on the fly without races ? And is it worth it in terms of performance to have one server instead of creating new ones ?