The study investigated the impact of query rewriting on Retrieval-Augmented Generation (RAG) systems. Four query-rewriting strategies (S1-S4) were compared against two strong LLM baselines (HyDE, Query2Doc) across three datasets: HotpotQA, AmbigNQ, and the 512K-document EnterpriseRAG-Bench. The research found that rewriting alone was at best competitive with a strong baseline. However, combining methods resulted in outsized gains, particularly when a post-hoc union of four methods (S1+S3+S4+HyDE) was used. This union improved HIT@10 over the baseline by +12.5 points on enterprise data.
The study highlighted the importance of complementarity. The union of five methods reached 52.98, and a confidence-gated router, applied via cost-aware routing, captured approximately half of the enterprise full-merge gain. This router reduced rewriting costs to <40% of queries and automatically declined rewriting on AmbigNQ. A downstream answer-quality evaluation using F1 scores confirmed the router’s effectiveness, improving F1 by +1.92 (p<0.01) at roughly 40% of the expansion cost.
Ultimately, the research suggests treating query rewriting as a complementary coverage source, applied through a cost-aware routing mechanism, rather than a standalone replacement for a strong baseline. This approach optimizes performance and resource utilization within RAG systems. Source: https://arxiv.org/abs/2609.05637v2