ActiveContext: Allow disabling ref fields

What does this MR do and why?

#536212 (closed) is split over a few MRs:

1. Allow disabling ref fields ⬅️ this MR
2. !190313 (merged): Reference, collection, queue, migration classes for Code collection
3. !190224 (merged): Standardised error handling for preprocessors

References

• Draft: Code embedding files using ActiveContext (!189310 - closed)
• [Embedding indexing pipeline] Reference class (#536212 - closed)

How to set up and validate locally

1. Check out 536212-3-code-embeddings-files which depends on this branch but contains Code files
2. Update the initializer config/initializers/active_context.rb by changing false to true
3. Create a connection
4. Run the migration worker: Ai::ActiveContext::MigrationWorker.new.perform
5. Verify that the partitions were created with the right schema (particularly without ref_id and ref_version)

# on ES
GET gitlab_active_context_code

# on psql
\d gitlab_active_context_code

8. Verify that the collection record exists and has the right values, particularly include_ref_fields is false

ActiveContext.adapter.connection.collections
=> [#<Ai::ActiveContext::Collection:0x0000000168a50c80
  id: 48,
  name: "gitlab_active_context_code",
  metadata: {"collection_class"=>"Ai::Context::Collections::Code", "include_ref_fields"=>false, "indexing_embedding_versions"=>[1]},
  number_of_partitions: 1,
  created_at: Tue, 06 May 2025 12:49:23.504247000 UTC +00:00,
  updated_at: Tue, 06 May 2025 12:50:37.082487000 UTC +00:00,
  connection_id: 10,
  include_ref_fields: false,
  indexing_embedding_versions: [1],
  search_embedding_version: nil,
  collection_class: "Ai::Context::Collections::Code">]

[Optional]

1. Delete the partitions
2. Set the migrations as not completed: ActiveContext.adapter.connection.migrations.update_all(status: 'pending')
3. Change the migration to create the collection by removing the options (thereby making include_ref_fields true)
4. Repeat steps 4-6 and this time see that ref_id and ref_version are present.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #536212 (closed)