Beyond Keywords: Measuring Multimodal Search Quality

Modern search systems return both text and images, requiring evaluation beyond traditional keyword matching. Our approach leverages embedding models to assess search quality across all modalities simultaneously. We present an open-source tool that quantifies semantic relevance between queries and multimodal results, enabling developers to measure and improve search experiences. This framework captures nuanced relationships between text and visual content, providing actionable metrics for optimizing multimodal search systems.

Philippe Bouzaglou

Vectra

While studying at Harvard alongside Mark Zuckerberg, Philippe played a pivotal role in shaping Facebook’s early days by introducing the concept of the Social Graph during the platform’s founding stage in the famous Harvard college dorm room. Since then, he has achieved numerous milestones in data science and machine learning, including serving as Vice President of Data Science and Machine Learning at Gorillas, a leading European quick-commerce food delivery company.