DEVOURED
From Scoring to Spelling: Rebuilding Ads Retrieval at Instacart
Instacart replaced its scoring-based BERT ads retrieval system with a generative model that predicts product recommendations token-by-token.
What: Instacart moved away from scoring every product ID via BERT, instead using a generative approach that outputs Instacart Semantic IDs. This resolved vocabulary constraints, structural drift, and cold-start issues for the ads platform.
Why it matters: This demonstrates a growing trend in retrieval systems where traditional scoring bottlenecks are bypassed by treating recommendation as a language generation task.
Deep dive
- Generation vs. Scoring: Eliminated the need to score the entire product catalog, significantly reducing latency.
- Semantic IDs: Utilizes an embedding-based identifier system to represent product features in a continuous space.
- Generalization: The generative approach allows for better session-based recommendations and handles new products more effectively.
- Performance: Solves the issue of structural drift where older models failed to account for changing inventory or user trends.
Decoder
- Retrieval: The phase of a recommendation system that quickly narrows down millions of items to a smaller set of candidates for ranking.
- Cold-start problem: The challenge of recommending items (or users) that have little to no interaction history.
- Structural drift: A phenomenon where a model's performance degrades over time because the underlying data distribution changes, causing old relationships to lose relevance.
Original article
Instacart rebuilt its ads retrieval system by moving from a scoring-based BERT model (that scored every product ID) to a generative approach that “spells out” recommendations token-by-token using Instacart Semantic IDs. This shift solved vocabulary bottlenecks, cold-start problems, and structural drift, enabling better generalization, full catalog coverage, and stronger session understanding.
Explore all 55 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi