[Embedding models] Use indexing_embedding_versions from collection
Implement the indexing changes for using indexing_embedding_versions to generate and set embeddings according to the figjam board.
A collection can have MODELS defined:
MODELS = { 0: { field:, model: } }
A collection record should include indexing_embedding_versions which is an array of ints (default to nothing).
During indexing, embeddings are generated for every indexing_embedding_versions and stored on the corresponding field.
-
Allow adding MODELSon a collection class -
Add a migration type to set indexing_embedding_versions. It should take an array of ints. -
Add a migration type to set search_embedding_version. It should take an int. -
Add documentation for how to set the initial embedding model -
Add documentation for how to switch to a new embedding model
Add migration to backfill data (#523204) is for the data backfill migrations and is out of scope of this issue.
Edited by Madelein van Niekerk