Our Sessions

Could AI code generation replace Learning to Rank? AI coding tools can generate rankers, but only up to a point. What techniques matter when building an agent coded ranker? And where do traditional search techniques still work?

We introduce “Learning to Understand” as a corollary to the well known “Learning to Rank” process. By using evals to learn domain-specific query interpretation and rewriting rules and combining with semantic statistics from your index, it’s possible to significantly improve search quality beyond typical BM25, vector, and hybrid search techniques.

Search relevance tuning is notoriously difficult, often requiring a deep understanding of Lucene scoring, complex query DSLs, and iterative manual testing. This session introduces Agentic Relevance Tuning—a framework that leverages LLM-based agents to automate the full search lifecycle making search tuning faster, more accurate, and accessible.

Rolling out semantic search is easy right? Just turn on some vectors and bim bam boom you have vector search... Right? Turns out when you're GitHub sized it's not quite that easy. We'll walk through the process we took, the lessons we've learned, and how you can build a plan to deploy vector search easier.

Traditional search pipelines rely heavily on static query parsing and after-the-fact relevance analysis. In this session, we present a new paradigm: using LangGraph with OpenSearch to create an agentic-based system that can tune hybrid search in real time.

Even though search teams are structured differently across the industry, they share common challenges like balancing learning with delivery and nurturing a culture built for continuous iteration. This talk distills a decade of organizational lessons from building Yelp’s AI-powered search into repeatable patterns for any team facing similar hurdles.

Lightning Talks

Closing

Welcome

In a world where AI shapes what we see, think, and do, true enginering lies not in using tools—but in designing them.

This talk uses a real-world B2B search case to show how a decision-based tree helps quickly diagnose why search fails and how to improve relevance without rebuilding the system.

In distributed search, BM25 returns different results across nodes because IDF and average document length vary with each node's corpus state. StableTfl replaces these with a term-length rarity heuristic, eliminating all corpus dependency. On 22 BEIR datasets, it retains ~90% of BM25's NDCG@10 while guaranteeing identical rankings across nodes.

Search has a new User Interface! All search will be Agentic/RAG and delivered through a chat interface! The days of the monolithic search engine are over!

Purpose-built rerankers are faster and cheaper, but are they better? We argue LLM rerankers win on what matters most in production: instruction-following and iteration speed, with more-than-acceptable tradeoffs on cost and latency. Our discussion is backed by a case study from Superhuman's production hybrid email search system.

As search powers RAG and agentic systems, relevance goals shift from ranking documents to assembling effective context. This talk explores how traditional lexical, semantic, and hybrid relevance changes when feeding LLMs, with lessons on chunking and snippet extraction, diversification, evaluation, and more.