Search relevance pipelines at Shipt - machine learning, query understanding and ranking
Dipak Parmar and Bart Masters • Location: Theater 5 • Back to Haystack 2020
Search relevance pipeline consists of machine learning, query understanding processes, and customized ranking with historical engagement info. Query understanding is the process of inferring the intent of the search keyword by extracting semantic meaning from the search query.
Query understanding methods generally take place before the search engine retrieves and ranks results. It’s all about what happens before the search query is generated. Customized ranking is derived with scores like textual relevancy, past engagement score, past conversions, and geographical trends. Machine learning allows for the scalable combination of all these signals to find the globally maximized optimization of relevance and conversion.
In this presentation, we will talk about each of the above steps and how they can be helpful to improve the overall search experience. We will also discuss architectures behind building relevancy pipeline and services with Elasticsearch as a search engine. We will also give an overview of how Shipt’s search platform is developed as a case study.
Dipak Parmar is a Principal Search Engineer at Shipt Inc. He has over 14 years of experience in software development including 8 years in implementing search using Apache Solr / Elasticsearch, and played lead/architect roles. He implemented custom search relevancy & ranking algorithms, query understanding, near real time indexing pipeline, search cluster & cloud design, and worked on performance tuning aspect of the search. In the past, he worked with many e-Commerce retailers in building search platform and micro-services. He lives in San Francisco Bay Area with his wife and a 4 year old kid.
Bart Masters leads a data science team at Shipt that focuses on building model-based solutions for Shipt's search, shopper, and catalog departments. Bart brings advanced analytics and data science experience from the healthcare, biosciences, and engineering industries. He lives in Birmingham, AL with his wife, 4 kids, and dog.