HTTPX::ConnectTimeoutError on v1.4.0
We are using HTTPX to make 500 concurrent HTTP/2 requests to Firebase's API. We build the requests as follows:
HTTPX
.plugin(:auth)
.plugin(:retries, max_retries: 3, retry_after: <exponent>, retry_change_requests: true)
.bearer_auth(access_token)
.with(headers: { 'Content-Type' => 'application/json', 'Accept' => 'application/json' })
.request(['POST', url, {body: payload, timeout: {connect_timeout: 10, read_timeout: 30, write_timeout: 30}]...)
We upgraded to v1.4.0 and suddenly started seeing a huge spike of HTTPX::ConnectTimeoutError errors. Please find the details of the message below:

We see that the new httpx version changed its architecture to use per session connection pools. It is also specified that connections can be reused. Because of that, we deduced that the code to compute the timeout is called more often, which in turn uses the default values of 2 seconds: ref.
In order to fix this, we increased the resolve_timeout to 5 seconds. But we still had this issue. Could you please let us know how we can debug this?
We are not able to find any APM traces in Datadog with this error. It looks like only request/responses with some http status is reported in Datadog.