Stop storing externally stored payloads that hold nothing
What does this MR do and why?
lib/gitlab/externally_stored_field.rb stores certain model fields as a JSON document in object storage instead of in Postgres. Two models use it: WorkItems::AgentPlan (fields content and readiness_score_feedback) and Ci::Catalog::BundledResources::Version (field readme).
These fields also go through CacheMarkdownField. Markdown caching renders a nil markdown field to an empty string for the matching _html field, and writing that empty string counts as an attribute change. The concern writes the JSON document whenever any externally stored field changed, so a record created with no content at all still triggered a write. The result was a document like {"content":null,"content_html":""} in object storage.
In practice, every GraphQL workItemEnableAiPlanning call creates a work_item_agent_plans row before any plan content exists, so each call wrote an empty document. Those objects are also swept into instance backups.
The fix is about 20 lines:
persist_external_payloadnow skips storing when every value in the payload is blank, and calls a newremove_external_payloadinstead.remove_external_payloadchecksexternal_storage_uploader.file&.exists?before callingremove!. The guard is needed because most records reaching this branch never stored anything, and the Fog object storage backend raisesNoMethodErrorinstead of reporting a missing object.
So records with no content store nothing, and clearing the last populated field deletes the document rather than leaving an empty one behind.
This was deliberately split out of !251354 so the storage waste can be fixed on its own. That merge request also contains this fix, so it will conflict trivially on this file once this one merges.
Tests: 5 new examples in spec/lib/gitlab/externally_stored_field_spec.rb under a blank payloads describe block. They use a test model with cache_markdown_field :content, storage: :external, because markdown caching is what makes the bug reachable. 2 of the 5 fail without the fix. The full file is 31 examples, 0 failures with the fix. RuboCop is clean.
References
- Requested by a reviewer as step 1 of a larger plan: !251354 (comment 3862419390)
- The larger change this was split out of: !251354
- Related issue: #616229
Screenshots or screen recordings
There is no UI change, so no screenshots apply.
How to set up and validate locally
- Run
bundle exec rspec spec/lib/gitlab/externally_stored_field_spec.rb. - To see the bug, enable the
workplanfeature flag for a group. In a Rails console, call theworkItemEnableAiPlanningGraphQL mutation on a work item, then look for a JSON file under the agent plan content object storage path (shared/agent_plan_content/locally). Before this change a file appears containing{"content":null,"content_html":""}. After it, no file is written.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist.