Support consolidated object storage configuration for agent plans

Asherah Connorrequested to merge
push-toqkvqtqxupv into master

What does this MR do and why?

Supports consolidated object storage configuration for agent plans, not just legacy (per-bucket).

References

Closes Agent plan storage needs to support consolidate... (#600491 - closed).

How to set up and validate locally

  1. Check out the branch.

  2. We need object storage enabled in GDK, and to be using consolidated object storage configuration. Here's the GDK docs on object storage. This should be enough:

    $ gdk config set object_store.enabled true
$ gdk config set object_store.consolidated_form true
$ gdk reconfigure

  3. Now you're configured to use MinIO for object storage and the consolidated configuration format (which this MR is adding support to agent plans for; see Create WorkItem::AgentPlan and its table, with ... (!232335 - merged) which added the initial per-bucket support only). The GDK probably doesn't know about agent plans yet (unless Add agent_plan_content config and MinIO bucket (gitlab-development-kit!5970 - merged) is merged and you have the latest GDK as of review), so you'll need to head into gitlab/config/gitlab.yml, and add it.

    There's a part that should look somewhat like this as of your gdk reconfigure:

    object_store:
  enabled: true
  proxy_download: false
  connection: {"provider":"AWS",...,"endpoint":"http://172.16.123.1:9000","path_style":true}
  objects: {"artifacts":{"bucket":"artifacts"},...,"gitaly_bundles":{"bucket":"gitaly-bundles"}}

    Add a new entry to objects, so it ends in ...,"agent_plan_content":{"bucket":"agent-plan-content"}. Note the underscores in the key name, but hyphens in the bucket name.

    Don't gdk reconfigure after this point until you're ready to lose that manual edit.

  4. Verify Rails has picked up the new configuration by checking at bin/rails console. Here's me verifying it:

    --------------------------------------------------------------------------------
 Ruby:         ruby 3.3.11 (2026-03-26 revision 1f2d15125a) [arm64-darwin24]
 GitLab:       19.1.0-pre (1fb6c9f5d7e) EE
 GitLab Shell: 14.51.0
 PostgreSQL:   17.8
--------------------------------------------------------------------------------
Loading development environment (Rails 7.2.3)
[1] pry(main)> Settings.agent_plan_content["object_store"]
=> #<GitlabSettings::Options:0x0000000135bb5950
 @options=
  {"enabled"=>true,
   "connection"=>
    #<GitlabSettings::Options:0x00000001357d1140
     @options=
      {"provider"=>"AWS",
       "aws_access_key_id"=>"minio",
       "aws_secret_access_key"=>"gdk-minio",
       "region"=>"gdk",
       "endpoint"=>"http://172.16.123.1:9000",
       "path_style"=>true}>,
   "proxy_download"=>false,
   "direct_upload"=>true,
   "storage_options"=>#<GitlabSettings::Options:0x0000000135bb59a0 @options={}>,
   "remote_directory"=>"agent-plan-content",
   "bucket_prefix"=>nil,
   "consolidated_settings"=>true}>

    Here's the same check with the same config file on master. If you see this, something or someone is being naughty:

    --------------------------------------------------------------------------------
 Ruby:         ruby 3.3.11 (2026-03-26 revision 1f2d15125a) [arm64-darwin24]
 GitLab:       19.1.0-pre (2e3f877d8dc) EE
 GitLab Shell: 14.51.0
 PostgreSQL:   17.8
--------------------------------------------------------------------------------
Loading development environment (Rails 7.2.3)
[1] pry(main)> Settings.agent_plan_content["object_store"]
=> #<GitlabSettings::Options:0x00000001367c4840
 @options=
  {"enabled"=>false,
   "remote_directory"=>nil,
   "bucket_prefix"=>nil,
   "direct_upload"=>true,
   "proxy_download"=>false,
   "storage_options"=>#<GitlabSettings::Options:0x00000001367c47c8 @options={}>}>

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Asherah Connor

Merge request reports