Learning to Rank in an Hourly Job Marketplace

As the largest online marketplace for hourly jobs in the US, Snagajob strives to connect millions of job seekers with part/full time, hourly and on-demand employment opportunities on a daily basis. Satisfactory fulfilment of this mission requires a two-way 'match' engine that can efficiently identify both the most suitable jobs for job seekers and the most qualified candidates for employers. In 2017, we reached an important milestone towards our vision of the match engine with a completely redesigned learning-to-rank (LTR)-based search solution that has given us significant lift in both information retrieval (IR) metrics such as NDCG@10 and key real-world business metrics like application rates.

Building an LTR solution for an hourly job marketplace entails its own set of interesting challenges. In this talk, we will discuss our technical journey in the past 12 months and bring plenty of in-the-trench stories where we succeeded (or failed) at wrangling our search system. Topics covered will include: 1) our selection of search engine technology, machine learning model and data engineering pipeline. 2) special characteristics of LTR and the job-search use case and implications for commonly used LTR features (e.g. BM25 scores) and machine learning techniques. 3) testing, monitoring and relevancy- tuning (yes, you still have to do a bit of that) the LTR model once it's online. 4) previews of a few modeling and engineering enhancements to the LTR solution and the match engine such as semantics features, live feature logging, personalization and potential integration with AWS Sagemaker.