Stop storing externally stored payloads that hold nothing

What does this MR do and why?

lib/gitlab/externally_stored_field.rb stores certain model fields as a JSON document in object storage instead of in Postgres. Two models use it: WorkItems::AgentPlan (fields content and readiness_score_feedback) and Ci::Catalog::BundledResources::Version (field readme).

These fields also go through CacheMarkdownField. Markdown caching renders a nil markdown field to an empty string for the matching _html field, and writing that empty string counts as an attribute change. The concern writes the JSON document whenever any externally stored field changed, so a record created with no content at all still triggered a write. The result was a document like {"content":null,"content_html":""} in object storage.

In practice, every GraphQL workItemEnableAiPlanning call creates a work_item_agent_plans row before any plan content exists, so each call wrote an empty document. Those objects are also swept into instance backups.

The fix is about 20 lines:

  • persist_external_payload now skips storing when every value in the payload is blank, and calls a new remove_external_payload instead.
  • remove_external_payload checks external_storage_uploader.file&.exists? before calling remove!. The guard is needed because most records reaching this branch never stored anything, and the Fog object storage backend raises NoMethodError instead of reporting a missing object.

So records with no content store nothing, and clearing the last populated field deletes the document rather than leaving an empty one behind.

This was deliberately split out of !251354 so the storage waste can be fixed on its own. That merge request also contains this fix, so it will conflict trivially on this file once this one merges.

Tests: 5 new examples in spec/lib/gitlab/externally_stored_field_spec.rb under a blank payloads describe block. They use a test model with cache_markdown_field :content, storage: :external, because markdown caching is what makes the bug reachable. 2 of the 5 fail without the fix. The full file is 31 examples, 0 failures with the fix. RuboCop is clean.

References

Screenshots or screen recordings

There is no UI change, so no screenshots apply.

How to set up and validate locally

  1. Run bundle exec rspec spec/lib/gitlab/externally_stored_field_spec.rb.
  2. To see the bug, enable the workplan feature flag for a group. In a Rails console, call the workItemEnableAiPlanning GraphQL mutation on a work item, then look for a JSON file under the agent plan content object storage path (shared/agent_plan_content/ locally). Before this change a file appears containing {"content":null,"content_html":""}. After it, no file is written.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Merge request reports

Loading
Loading