Learning to Rank in an Hourly Job Marketplace

Xun Wang • Back to Haystack 2018

As the largest online marketplace for hourly jobs in the US, Snagajob strives to connect millions of job seekers with part/full time, hourly and on-demand employment opportunities on a daily basis. Satisfactory fulfilment of this mission requires a two-way 'match' engine that can efficiently identify both the most suitable jobs for job seekers and the most qualified candidates for employers. In 2017, we reached an important milestone towards our vision of the match engine with a completely redesigned learning-to-rank (LTR)-based search solution that has given us significant lift in both information retrieval (IR) metrics such as NDCG@10 and key real-world business metrics like application rates.

Building an LTR solution for an hourly job marketplace entails its own set of interesting challenges. In this talk, we will discuss our technical journey in the past 12 months and bring plenty of in-the-trench stories where we succeeded (or failed) at wrangling our search system. Topics covered will include: 1) our selection of search engine technology, machine learning model and data engineering pipeline. 2) special characteristics of LTR and the job-search use case and implications for commonly used LTR features (e.g. BM25 scores) and machine learning techniques. 3) testing, monitoring and relevancy- tuning (yes, you still have to do a bit of that) the LTR model once it's online. 4) previews of a few modeling and engineering enhancements to the LTR solution and the match engine such as semantics features, live feature logging, personalization and potential integration with AWS Sagemaker.

View the Slides
Xun Wang

Xun Wang is a Principal Data Scientist from Snagajob’s Match team at its DC office. Since joining Snagajob in 2016, Xun has been the data science and machine learning engineering lead for Snagjob’s next-generation learning to rank search products learning, and has contributed to a variety of projects regarding text mining, forecasting, recommendation and data-driven decision making in general.

Prior to Snagajob, Xun was a Senior Data Scientist at Sentrana Inc, an early pioneer in large-scale predictive analytics with applications in sales and marketing. Xun holds a Ph.D. in Transportation Systems Engineering from Cornell University. Before coming to the US in 2008, Xun spent the first 22 years of his life in his beloved hometown Nanjing, China.