Iterative Agent Loop Outperforms Traditional RAG Pipelines on Google FRAMES Benchmark
Developers benchmarked 18 traditional Retrieval-Augmented Generation (RAG) pipeline variants against an iterative agent loop across 824 multi-hop questions in Google's FRAMES dataset. The agentic retrieval loop achieved 92.7% accuracy, significantly outperforming the best traditional pipeline variant which reached 78.9%. The results demonstrate that dynamic, multi-step agent loops are far more effective for complex reasoning tasks than static pipeline optimizations like query expansion or reranking. This highlights a technical shift toward agentic architectures for enterprise information retrieval and question-answering systems. Unexpectedly, adding a small reranker reduced the top pipeline's accuracy by 9 percentage points, while a larger reranker provided minimal performance gains. Additionally, the benchmark revealed that models often answered using parametric memory despite strict instructions to rely solely on retrieved context, requiring manual verification of all correct answers.
## BACKGROUND
Traditional RAG relies on a static single-step pipeline—embedding a prompt, retrieving matching document chunks, and generating an answer—which struggles with multi-hop reasoning where facts must be combined from multiple sources. Google's FRAMES benchmark tests end-to-end retrieval and reasoning across 2 to 15 Wikipedia articles. Agentic RAG replaces static pipelines with a reasoning loop where the LLM dynamically evaluates retrieval results and issues follow-up queries as needed.