subhopam das closed issue #6: Improve RAG Accuracy: Tune Weights, Ranking, and Model Parameters at GitLab AI Hackathon / Participants / kapillamba4

subhopam das
2026-03-20T01:41:08Z

Overview

Improve the overall accuracy of the RAG (Retrieval-Augmented Generation) pipeline by systematically tuning retrieval and generation components.

Goals

Improve retrieval precision and recall by tuning chunk sizes, overlap, and embedding models

Optimize ranking/re-ranking strategies (e.g., BM25, cross-encoder re-rankers, MMR)

Tune similarity score thresholds and top-k retrieval parameters

Experiment with different embedding models for better semantic representation

Evaluate and improve the LLM prompt templates for better answer generation

Implement and benchmark hybrid search (dense + sparse retrieval)

Tasks

Baseline evaluation: measure current accuracy metrics (e.g., RAGAS, MRR, NDCG)

Experiment with chunk size and overlap tuning

Evaluate alternative embedding models

Implement re-ranking (e.g., cross-encoder or Cohere re-rank)

Tune top-k and similarity threshold parameters

Explore hybrid search (BM25 + vector search)

Improve prompt engineering for the generation step

Run A/B evaluations and document results

Update configuration with best-performing settings

Acceptance Criteria

Measurable improvement in RAG accuracy metrics over the baseline

Documented evaluation results comparing different configurations

Updated and reproducible configuration for the best-performing setup