Improving Relevance in RAG - Lessons learned from the LiveRAG challenge

Any successful Retrieval Augmented Generation (RAG) system hinges on its ability to retrieve relevant context. This talk presents key lessons learned from our participation in this year’s SIGIR LiveRAG challenge, where we were tasked with building a RAG system to generate accurate answers under time constraints. We will deconstruct the RAG pipeline we developed, focusing on our journey to improve the retrieval component. Specifically, we’ll highlight how synthetic data generation was crucial in the evaluation process. Beyond the theoretical, we’ll also share practical insights into the engineering challenges of running a large-scale RAG system locally and the solutions we implemented to overcome them. This talk is for anyone interested in building more robust and reliable RAG applications.

Matthias Krüger

OpenSource Connections