Session Description

We trained an open image-text embedding model in hyperbolic space. It keeps the strengths of vector search while adding hierarchy to the embedding itself, so not every visual relationship has to be pre-written into structured data or a knowledge graph.

Product catalogs are hierarchical: categories contain product families, product families contain variants, and variants contain visual details. The hard part is that catalog teams cannot predict every way a user will describe a specific item. A shopper might type “white polo shirt with a small crocodile logo” when they want the product behind that visual cue, even if no one maintained “crocodile logo” as a tag. Manual tags, rules, and knowledge graphs can help, but they are expensive to maintain and impossible to complete.

In this model, hierarchy is part of the geometry. Hyperbolic radius gives a notion of specificity: broader concepts sit closer to the center, more specific ones farther out. The model is trained using hierarchical image-text data rather than hand-written catalog labels.

The talk includes demos, failure cases, and measurements on public data with released weights and rerunnable evals: caption hierarchies, parent-child retrieval controls, in-shop product retrieval, and negative results where the approach does not help. The stack includes an MIT-licensed open-source implementation and open weights, so the model can be adapted further for domain-specific catalogs.

Outline (45 minutes including Q&A):

1. What hyperbolic image-text embeddings add to visual search (6 min)
2. Product-search examples where structured data and tags are not enough (8 min)
3. Training from hierarchical image-text data (7 min)
4. Demos and evaluation results: caption hierarchy, product retrieval, parent-child controls, and failure cases (17 min)
5. Integration patterns for existing vector-search stacks and domain adaptation (7 min)