Add Gitlab::BufferedIo with header read timeout
What does this MR do and why?
This adds a timeout for reading HTTP response headers. It is needed because headers are fetched before a read_timeout comes into effect.
Related to: #336999 (closed) (confidential)
How does it work
Headers are read using BufferedIO#readuntil (see lib/net/http/response.rb:57). It loops through each block that was sent by the server until it finds a new line /n. The patch hooks into this and checks the time in each loop. If the time is over, it raises a Gitlab::HTTP::HeaderReadTimeout error.
Risks
BufferedIO#readuntil is a generic method that is not dedicated to read header data. Overriding it can have unwanted side effects that are hard to spot. Since it is in a Ruby core library, that is also used by the Rails framework and other Gems, it has a far-reaching impact. This was discussed here (confidential).
To prevent side effects, the patch:
- is behind the header_read_timeout_buffered_io feature flag
- is only used when a request is made using the use_read_total_timeout option
Screenshots or screen recordings
How to set up and validate locally
See #336999 (comment 808312938) (confidential)
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.