From BM25 to Mixture-of-Encoders: Evaluations for Next-Gen Search and Retrieval Systems

Filip Makraduli • Location: TUECHTIG • Back to Haystack EU 2024

Modern user queries require a mix of structured and unstructured data in order to achieve satisfactory retrieval performance. This is where traditional search methods fall short. In this talk, we dive into retrieval evaluation, comparing keyword, vector, hybrid, and late-interaction models with Superlinked’s mixture-of-encoders approach. We examine how each approach fares in real-world scenarios (e.g. a query for “5 guests under $200 with 4.8+ rating”). Using benchmark datasets and real production use cases, we share metrics, evaluation methodology, and common pitfalls. We introduce Superlinked’s mixture-of-encoders approach, where dedicated encoders for various data types like text, numbers, and categories combined with LLM-driven query understanding enable more accurate and scalable retrieval. Finally, we discuss how to productionize this system and share use cases from travel to e-commerce, pointing toward the future of multi-attribute and meta data aware embeddings search.

Filip Makraduli

Superlinked

Filip Makraduli is a machine learning engineer with a strong background in AI systems, vector search, and large language models (LLMs). He holds a Master’s degree in Biomedical Data Science from Imperial College London. Currently, Filip works as a Founding Developer Relations Engineer at Superlinked, where he focuses on integration engineering, deep tech content creation, and developer advocacy. His work emphasizes the use of multi-encoder architectures to enhance retrieval quality and reduce reliance on reranking strategies. In the past, Filip worked as a data scientist at Marks & Spencer, where he contributed to AI-driven solutions for retail. He has also held machine learning engineering roles across several UK-based startups, focusing on applied AI and product-oriented ML development. In addition to his industry work, Filip has been active in the open-source community, particularly around LLM tooling and pipelines. He has delivered various talks on practical machine learning applications.