Haystack 2018

How about I run either a panel session or a fishbowl session (where panel members can be replaced with audience members). The general theme would be to tell stories of crazy search situations we've all found ourselves in, naming no names, and use these to illustrate why relevance tuning is hard, highly specific to the industry and situation you find yourself in, and often totally misunderstood by management, marketing etc. More »

As the largest online marketplace for hourly jobs in the US, Snagajob strives to connect millions of job seekers with part/full time, hourly and on-demand employment opportunities on a daily basis. Satisfactory fulfilment of this mission requires a two-way 'match' engine that can efficiently identify both the most suitable jobs for job seekers and the most qualified candidates for employers. In 2017, we reached an important... More »

Many online shops around the world use open source technologies like Solr and Elasticsearch for their onsite search. However, the underlying relevance scoring algorithms were largely developed with enterprise search or general site search in mind. Relatively little is known about the specifics of search relevance scoring in e-commerce search, even from the perspective of information retrieval theory. More »

Using a large sample of user searches and the documents that were engaged from those searches, LexisNexis set out to predict and re-rank the top answers so that engaged documents would appear closer to the top of the Search Engine Result Page (SERP). Lexis applied a three phased approach to collect and compile LETOR data to drive a Java LambdaMART implementation that re-ranked the top answers in... More »

10,000 people share my full name. How can I search information about me and not the other 9,999? Entity resolution gives you the power to disambiguate this search. It's a way to find different records about the same thing, while excluding similar records about different things. And it has an amazing ability to track the changes of an identity, like when you change your name or address. More »

An introduction to Vespa the "open big data serving engine" developed by oath.com, formerly Yahoo!. More »

The topic of Synonyms in Solr has been historically a charged topic, with on-going and lively discussions within the search community about various aspects of synonyms usage such as index- vs query-time trade-offs, tokens graphs challenges (a.k.a., "sausagization") with multi-term synonyms, and various parsed query outcomes to name just a few examples. In this talk, we will review and summarize the current state of the "synonyms art" in Solr... More »

Named entity recognition (NER) can identify search terms as fields or enum values in an index such as brand and location. The identified fields and enum values can be used as additional relevance signals or filters. This talk will demonstrate how to improve relevance for structured or semi-structured data search using NER. It will include practical lessons learned when applied to real-world datasets such as match vs term queries, data formatting, and the minimum training set size. More »

The talk covers the evolution of an Image Similarity Search Proof of Concept built to identify similar medical images. It discusses various image vectorizing techniques that were considered in order to convert images into searchable entities, an evaluation strategy to rank these techniques, as well as various indexing strategies to allow searching for similar images at scale. More »

You've spent months tuning outstanding results by any objective measure, and now you've got to mess that all up so that some new product type shows up in every search. In this presentation we'll discuss ways to handle several situations like that. Almost as important, we'll factor the impact into our analytics so that we can optimize both IR metrics and whatever it was Marketing was trying to accomplish. More »

The Semantic Knowledge Graph is an Apache Solr plugin that can be used to discover and rank the relationships between any arbitrary queries or terms within the search index. It is a relevancy swiss army knife, able to discover related terms and concepts, disambiguate different meanings of terms given their context, cleanup noise in datasets, discover previously unknown relationships between entities across documents and fields, rank lists of keywords... More »

Search is on the move. After decades of relying on counting terms, we observe a paradigm shift. As defining relevancy has become increasingly difficult, with the presence of a multitude of diverse and sometimes intertwined signals, new challenges for search engine practitioners call for new ways of thinking about search. Machine learning is such a new way of thinking, and it has become increasingly popular for solving search problems. More »

Word Embeddings have been providing the semantic glue in major search engines for the past several years (word2vec, GloVe, fastText, and Gensim have received a lot of press). This talk will provide concrete resources and examples to help you bootstrap distributional semantics into your search solutions and start improving recall for your long-tail queries. More »

Reliable text extraction is essential for search. Without reliable text, users cannot find the content they need, and advanced relevance tuning is worthless. Apache Tika™ is a crucial component in a wide variety of search engines and content management systems, including Solr™, Nutch™, Alfresco, Elasticsearch and Sleuth Kit®/Autopsy®. More »