### Vectorize content and code with Embeddings By creating [embeddings](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings) for content items (issues, epics, MR's) and storing them with [pgvector](https://github.com/pgvector/pgvector) we can work with larger datasets to Search, Cluster, Recommend, Anomaly detection, diversity measurement, classification. Especially [Backlog cleanup](https://gitlab.com/gitlab-org/gitlab/-/issues/393884#note_1300546991), deduplicaiton would become possible For the below tasks we would need to first build a scheduled vectorizer, which can chunk text, create embeddings and save them. As soon as this exists it can be reused in below tasks and complexity decreases from there. * Semantic/Contextual Search (2 - [Semantic Search](https://github.com/openai/openai-cookbook/blob/main/examples/Semantic_text_search_using_embeddings.ipynb)) * Q/A for knowledge base (2 - Ask the tanuki or [QAEmbeddings](https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb)) * Duplication check enhancement (2 - [Embedding Recommendation](https://github.com/openai/openai-cookbook/blob/main/examples/Recommendation_using_embeddings.ipynb) * Classification (2 - [Classification with Embeddings](https://github.com/openai/openai-cookbook/blob/main/examples/Classification_using_embeddings.ipynb)) * Cluster content (2 - [Clustering](https://github.com/openai/openai-cookbook/blob/main/examples/Clustering.ipynb)) * Deduplication (2) * [Suggest issues to relate to MRs](https://gitlab.com/gitlab-org/gitlab/-/issues/393884#note_1300546991)