Jmeter test always freeze when tested server gives up

When trying to run load test in JMeter 5.1.1, tests always freeze at the end if server gives up. Test completes correctly if server does not give up. Now this is terrible because the point of test is to see at what point server gives up but as mentioned test never ends and it is necessary to kill it by hand.

Example:

• Test running 500 threads for local server goes smoothly and it finish with tiding up message
• Exactly the same test running 500 threads for cloud based server at some points results in error test goes to about 99 % then freezes on summary as in below example:

summary + 99 in 00:10:11 = 8.7/s Avg: 872 Min: 235 Max: 5265 Err: 23633 (100.00%) Active: 500 Started: 500 Finished: 480

and that's it you can wait forever and it will just be stuck at this point.

Tried to use different thread types without success. Next step was to change Sampler error behavior and yes changing it from Continue to Start Next Thread Loop or Stop thread helps and test is ending but then results in html look bizarre and inaccurate. I even tried to set timeout setting to 60000 ms in HTTP request Defaults but this also has given strange results.

That said can someone tell me how to successful run load test for server so that is always completes regardless of issues and is accurate> Also I did see few old question about the same issue and they did not have any answer that would be helpful. Or is there any other more reliable open source testing app that also has GUI to create tests?