Skip to content

Basic support for GitLab Geo transfers

Stan Hu requested to merge geo-transfer-support into master

How this works:

  1. A Geo transfer request arrives with a JWT header with the right data (e.g. URL =/api/v3/geo/transfers/lfs/1 for LFS object ID 1, with a JWT token that includes the corresponding LFS OID)
  2. Workhorse proxies the request and the Rails backend verifies the validity of the request.
  3. If the request is valid, the Rails backend uses X-Sendfile functionality in Workhorse/nginx to send data back to the client.

Current Geo Nodes use the system hook token for authentication, which is not that secure. This implementation creates an access identifier and an secret access key for each GeoNode. The GeoNode uses that to create a JWT token in the Authorization header. The secret access key is encrypted with the db_key_base valid and replicated in PostgreSQL. Since db_key_base has to be correct to decode this field, we are ultimately relying on the security of that key.

The primary GeoNode receives the Authorization header, looks up the proper GeoNode with the access identifier, and then validates the JWT token. We expect that the times of the nodes are synchronized within 1 minute to prevent replay attacks.

The MR to schedule the downloads will come in a separate MR. That currently depends on !1197 (merged) being merged and some work in a personal branch.

Merge request reports