Making the case for human judgement relevance testing

View the Slides View the Video

Supporting a robust relevance testing programme using human judges represents a significant investment in terms of time and resources. However, when executed well the outputs can quickly become an invaluable tool and important motivator not just for search relevance engineers, but for product teams and other business stakeholders.

This talk will explore what makes a successful human relevance testing programme that ultimately speaks for itself in terms of return on investment. It will cover:

• Establishing a process with sufficient rigor to deliver reliable and actionable quantitative and qualitative findings across a range of study types.
• Analysis of results and using insights to drive business recommendations.
• Using human relevance testing to complement and augment insights from other search testing methods such as product analytics.
• Effective communication of results and insights to technical and business audiences.

Tara Diedrichsen

LexisNexis

Tara Diedrichsen is a Senior Global Product Manager in the Global Product Team at LexisNexis. She works with product teams around the global on search optimization, including the adoption of new search algorithms as well new metrics and testing methods. In this capacity she manages the human judgement relevance testing program at LexisNexis globally.