Fix flaky tests of the TestGitalyServerFactory test suite

The test is flaky for a quite some time already. It is a next attempt to fix it.
The change contains a couple of small fixes:

• removal of the LimiterMiddleware that shouldn't took part in the test
• adding of the missing Stop() calls for the factories to terminate all running servers
• removal of the Close() calls for the listeners as they will be terminated by the gRPC server
• await for the termination of all running goroutines before test completion

Closes: #3916 (closed)