ActiveContext: update embeddings request batch size in Service classes
What does this MR do and why?
We need to persist a batch size value for Semantic Search indexing. This value indicates the maximum number of individual content inputs for each bulk embeddings request. Currently, we have set this to 30 for all embedding models, but Self-Managed instances with their own Self-hosted models will need to specify their own batch size since this value depends on the selected model.
Change Steps
- Introduce
embeddings_request_batch_sizeto the embeddings model metadata schema - !246671 (merged) - Make the batch size value editable from the Admin UI for Self-Managed instances. This change has to be split in 2 parts due to its size
- Introduce or update
Serviceclasses to allow for batch size update - this MR - Allow batch size update from the Admin UI - !246635 (merged)
- Introduce or update
In this MR
- Update the
EmbeddingModelActivationServiceto accept anembeddings_request_batch_sizeparameter. This allows the batch size to be set when an embedding model is changed- Includes minor update to
SemanticSearchEmbeddingsControllerand spec file
- Includes minor update to
- Introduce a new service for updating the batch size only (
UpdateEmbeddingsRequestBatchSizeService). This allows the batch size to be updated by itself, without changing the model and requiring an expensive backfill process
References
- Issue: [Semantic Search] Persist batch_size for indexing (#600634 - closed)
- MR for Admin UI changes: Allow SM Admins to update embeddings request ba... (!246635 - merged)
Screenshots or screen recordings
N/A
How to set up and validate locally
Both services update DB records. The new optional batch size parameter for EmbeddingModelActivationService is not yet used, while the new service UpdateEmbeddingsRequestBatchSizeService is not used at all.
This has no production impact yet, and the unit tests should cover the required validation.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #600634 (closed)