Haystack Agenda

Talks from the Search & Relevance Community at the Haystack Conference!

Day 1 At-a-Glance

(scroll down for a detailed schedule)

8:00-9:00am Registration and Continental Breakfast
9:00-9:45am Opening Keynote
10:00-10:45am Concurrent Session 1
11:00-11:45am Concurrent Session 2
11:45am-1:15pm Lunch (on your own)
1:15-2:00pm Concurrent Session 3
2:15-3:00pm Concurrent Session 4
3:15-4:45pm Lightning Talks
5:30-6:30pm Reception (optional, included)
6:30-8:00pm Dinner (included)
8:00-9:00
Registration and Continental Breakfast
9:00-9:45
Keynote
Track One Track Two
10:00-10:45
Rated Ranking Evaluator: an Open Source Approach for Search Quality Evaluation

Alessandro Benedetti (Sease)

Every team working on Information Retrieval software struggles with the task of evaluating how well their system performs in terms of search quality... More »

Evaluation

Ontology and Oncology: NLP for Precision Medicine

Sean Mullane (University of Virginia)

This session gives an overview of the importance of precision medicine in cancer treatment and describes an approach used by UVA in the TREC 2018 Precision Medicine workshop. More »

NLP

11:00-11:45
Making the case for human judgement relevance testing

Tara Diedrichsen and Tito Sierra (LexisNexis)

Supporting a robust relevance testing programme using human judges represents a significant investment in terms of time and resources. However, when executed well the outputs can quickly become... More »

Evaluation

Analyzing Language Analyzers-Even in Languages You Don't Know

Trey Jones (Wikimedia Foundation)

Whether it's a minor upgrade from ASCII folding to ICU folding or deploying a completely new language analyzer, the effects of changes to your analysis chain can be hard to predict. More »

NLP

11:45-1:15
Lunch on your own
1:15-2:00
Towards a Learning To Rank Ecosystem @ Snag - We've got LTR to work! Now what?

Xun Wang (Snag)

As the largest online marketplace for hourly jobs in the US, Snag strives to connect millions of job seekers with part/full time, hourly and on-demand employment opportunities. More »

Learning to Rank

Query relaxation - a rewriting technique between search and recommendations

Rene Kriegler

In search quality optimisation, various techniques are used to improve recall, especially in order to avoid empty search result sets. More »

Query Rewriting

2:15-3:00
Evolution of Yelp search to a generalized ranking platform

Umesh Dangat (Yelp)

Elasticsearch forms the backbone of Yelp's core search. The Learning to Rank elasticsearch plugin is one of the key tools that has transformed the Yelp Search team... More »

Learning to Rank

Beyond The Search Engine: Improving Relevancy through Query Expansion

Taylor Rose and David Mitchell (Ibotta)

Due to a variable inventory and an ephemeral data set, users often search for terms that are outside of our corpus. This leads to empty search result sets, despite... More »

Query Rewriting

3:15-4:45
Lightning Talks

Informal, ad-hoc, spontaneous 5 minute lightning talks on search, relevance, information retrieval, and our community! We'll line up and count down from 5 minutes :)

5:30-6:30
Reception Optional
6:30-8:00
Dinner Included

Day 2 At-a-Glance

(scroll down for a detailed schedule)

8:00-9:00am Continental Breakfast
9:00-9:45am Concurrent Session 5
10:00-10:45am Concurrent Session 6
11:00-11:45am Concurrent Session 7
11:45am-1:15pm Lunch (on your own)
1:15-2:00pm Concurrent Session 8
2:15-3:00pm Concurrent Session 9
3:15-4:00pm Concurrent Session 10
4:00pm Conference Closing
8:00-9:00
Breakfast service
Track One Track Two
9:00-9:45
Addressing variance in AB tests: Interleaved evaluation of rankers

Erik Bernhardson (Wikimedia)

Evaluation of search quality is essential for developing effective rankers. Interleaved comparison methods achieve statistical significance with less data than... More »

Evaluation

How The New York Times Tackles Relevance

Jeremiah Via (The New York Times)

The New York Times has had search for a long time but 2018 was the year in which the company engaged with relevance in a deep way. The aim of this talk is to share... More »

Use Case

10:00-10:45
Solving for Satisfaction: Introduction to Click Models

Elizabeth Haubert (OpenSource Connections)

Relevance metrics like NDGC or ERR require graded judgements to evaluate query relevance performance. But what happens when we don't know what 'good'... More »

Evaluation

Establishing a relevance focused culture in a large organization

Tom Burgmans (Wolters Kluwers)

For a relevance engineer one of the most difficult tasks in the tuning process is to convince others in the organization that this is a joint effort. Even the brightest... More »

Use Case

11:00-11:45
Custom Solr Query Parser Design Option, and Pros & Cons

Bertrand Rigaldies (OpenSource Connections)

Does your search application include a custom query syntax with various search operators such as Booleans, proximity, term or phrase frequency, capitalization, quoted text... More »

Misc

Architectural considerations on search relevancy in the context of e-commerce

Johannes Peter (Media Markt Saturn)

With an increasing amount of relevancy factors, relevancy fine-tuning becomes more complex as changing the impact of factors produces increasingly more... More »

Use Case

11:45-1:15
Lunch on your own
1:15-2:00
Using LTR to Personalize Search Experience at Go-FOOD

Maulik Soneji (Go Jek)

GoFood, the food delivery product of Gojek is one of the largest of its kind in the world. This talk summarizes the approaches considered and lessons learnt during the design... More »

Learning to Rank

Search Logs + Machine Learning = Auto-Tagging Inventory

John Berryman (Eventbrite)

For e-commerce applications, matching users with the items they want is the name of the game. If they can't find what they want then how can they buy anything?! More »

NLP

2:15-3:00
'Relevant' Machine Translation with Learning to Rank

Suneel Marthi (Amazon Web Services)

Learning to Rank (LTR) has been used successfully in several areas of Information Retrieval and Search for constructing ranking models. One of the less explored... More »

Learning to Rank

Natural Language Search with Knowledge Graphs

Trey Grainger (Lucidworks)

To optimally interpret most natural language queries, it is necessary to understand the phrases, entities, commands, and relationships represented or implied within... More »

NLP

3:15-4:00
Search with Vectors

Simon Hughes (Dice Holdings Inc.)

With the advent of deep learning and algorithms like word2vec and doc2vec, vectors-based representations are increasingly being used in search to represent anything from documents... More »

Misc

Search-based recommendations at Politico

Ryan Kohl (Politico)

Over the past year, the POLITICO team has developed a recommendation system for our users, which recommends not only news content to read but also news topics to subscribe to. More »

Misc

4:00
Conference Wrap-up