Session Description

Gender bias in AI-powered search is easy to miss and hard to measure. When an LLM mediates between a user and a catalog, bias can enter at two layers: what gets retrieved and how the agent narrates it. No standard tooling exists to measure it in production.

We built a two-layer detector evaluating linguistic patterns in agent responses and stereotypical product selection in retrieval results. The linguistic side worked reasonably well. The retrieval side was a mess: you cannot tell the difference between an agent that is biased and a catalog that already skews gendered. Every pink product flagged as bias was just… the catalog.

Counterfactual testing turned out to be the fix. Swap one name, while everything else remains identical: “Hi, I’m Alice” vs “Hi, I’m Andrew.” If the agent responds differently, that difference is the agent’s doing, not the data’s. We built 400+ test cases across five bias types grounded in Grep-BiasIR, ran them across five LLMs, and found that even well-regarded models fail more often than one would expect.

Whether you are building agentic search or just starting to think about fairness in your pipelines, you will walk away with a concrete testing approach and a clearer picture of where current tools still fall short.