Flaky TestDomainResolverError race

I've seen an increase in race test failures for TestDomainResolverError recently, I think it's worth investigating a bit more.

Job #2647463990 failed for be06885f: