Skip to content

Dependency proxy not fetching from object storage

Summary

The dependency proxy feature does not read from object storage when configured to do so.

If you configure object storage for the dependency proxy, the blobs in the database always have file_store as null. This results in them being read from the fileystem, rather than from object storage.

Further, I've noticed that if direct_upload is false, the objects are also not uploaded to object storage. With it set to true, they are uploaded, but not used during download.

Steps to reproduce

  1. Configure dependency proxy with object storage enabled
  2. Leave direct_upload set to true, to repro the fetch error
  3. Be sure to add EE license with proxy support to your instance
  4. Create a group to use with your proxy
  5. Pull an image you don't have locally from the proxy
  6. Check that new content was added to your bucket
  7. Open the rails console and list your DependencyProxy::Blob
  8. Note that file_store: is nil

Example failure on next image download, caught from the helm charts (which don't have shared storage other than object storage):

Completed 500 Internal Server Error in 309ms (ActiveRecord: 4.6ms | Elasticsearch: 0.0ms | Allocations: 7950)
gitlab-webservice-6848684fdf-7wbq5 webservice   
gitlab-webservice-6848684fdf-7wbq5 webservice ActionController::MissingFile (Cannot read file /srv/gitlab/shared/dependency_proxy/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35/dependency_proxy/2/files/1/323d0d660b6a7da8df08a01dbc7250f38cfa2161db00c7c27c0b20be07a8236a.gz):
gitlab-webservice-6848684fdf-7wbq5 webservice   
gitlab-webservice-6848684fdf-7wbq5 webservice app/controllers/concerns/send_file_upload.rb:22:in `send_upload'
gitlab-webservice-6848684fdf-7wbq5 webservice ee/app/controllers/groups/dependency_proxy_for_containers_controller.rb:27:in `blob'

Example Project

Dependency Proxy MR being added to the helm charts encounters this error: gitlab-org/charts/gitlab!1464 (merged)

What is the current bug behavior?

Dependency proxy always records local file store to the database.

What is the expected correct behavior?

Dependency proxy should use the remote filestore when configured to do so.

Relevant logs and/or screenshots

My dependency proxy config:

dependency_proxy: 
    enabled: true   
    object_store:   
      enabled: true                     
      remote_directory: <redacted>
      direct_upload: true                    
      background_upload: false
      proxy_download: true
      connection:
        provider: AWS
        region: us-east-1
        aws_access_key_id: "<redacted>"
        aws_secret_access_key: "<redacted>"
        host: <redacted>
        endpoint: <redacted>
        path_style: true

Possible fixes

Likely, similar to the terraform state uploaders, we need to add a default store check, that uses the remote store when object storage is enabled. And also we need to have the file_store updated in the database after the store! method is fired. As seen in the terraform state model. These two things are missing from the dependency_proxy blob and uploader.