Codebase as chat context: Consider indexing repositories ad hoc
Background
The storage usage from the near-complete gitlab-org rollout for the codebase as chat context feature raises a question on the cost / storage requirements of the feature.
Proposal
I'd like us to consider (brainstorm) indexing repositories ad hoc. In that case, the first time the user asks a question about a repository (tool call), we'd schedule indexing and continuously update the embeddings after that.
We could store a timestamp like queried_at on the active context project record and stop indexing the project after certain time and potentially delete it from the collection completely if no one is asking questions about it for a prolonged period of time.
This should massively reduce the storage usage as well as compute resources to keep chunking code and generate embeddings.
@partiaga @tgao3701908 @terrichu @arturoherrero @ck3g WDYT? Should we consider this option?