Session Description

If your team is struggling to balance business rule demands, modern vector workflows, and AI-powered evaluation, you are not alone. Across the search industry, production systems are being forced to evolve into multi-consumer environments serving human and agentic searchers. At the same time teams are confronted with legacy systems and shifting organizational dynamics.
In our consulting work at OpenSource Connections, we see firsthand what search teams are wrestling with as they are tasked with building AI-ready retrieval systems.
This presentation shares grounded insights directly from client engagements:

• How engineering teams are executing search replatforming for AI and how that interacts with their organizational structure and processes.

• How a new demographic of builders views search as an API-driven building block for autonomous tools.

• Why deterministic query governance remains vital alongside probabilistic models to provide tuning knobs to the business and have a certain degree of control over a non-deterministic system.

We will also examine why Generative AI has made search evaluation significantly more challenging, despite LLMs as judges becoming a staple for scaling quality assessment and explore how these observations impact us as a community.