Geo: Switch blob download to use GitLab::HTTP to avoid llhttp-ffi issue

What does this MR do and why?

Adds an alternative HTTP backend for Geo blob downloads behind a feature flag (geo_blob_download_with_gitlab_http), to work around a crash/data corruption issue affecting Omnibus installs on Ubuntu 24.04 with kernel >= 6.8.

The existing BlobDownloader uses the http gem (httprb), which depends on llhttp-ffi for HTTP parsing via FFI callbacks. An interaction between rugged 1.9.0's native extension and Omnibus's bundled libffi 3.2.1 corrupts these FFI callbacks on newer kernels, causing blob downloads to either segfault or silently return incorrect HTTP status codes.

The new code path uses Gitlab::HTTP (HTTParty/Net::HTTP), which has no FFI dependency and is unaffected.

Root cause

  • rugged was upgraded from 1.6.3 to 1.9.0 in GitLab 18.10
  • Omnibus ships libffi 3.2.1, which uses legacy trampoline/closure mechanisms
  • On kernel >= 6.8, stricter memory protections break libffi 3.2.1's executable memory allocation after rugged's native extension loads
  • llhttp-ffi creates FFI closures via method(:callback).to_proc which rely on the now-corrupted trampoline memory
  • The gem upgrade in !229530 (merged) (llhttp-ffi 0.5.1) does not fix this — the same callback pattern is used
  • See #595139 for full investigation and customer reports.

References

!202690 (comment 2472924688)

How to set up and validate locally

  1. Setup Geo on Omnibus (Ubuntu 24.04.1 LTS, kernel 6.17.0-1010-gcp, running 18.10.1-ee)
  2. Create blob type (i.e. Upload) - observe sync failure/remains pending
  3. Enable FF in this MR via the rails console and patch the modified file then restart sidekiq

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #595139

Edited by Scott Murray

Merge request reports

Loading