Democratizing Machine Learning at Netflix: Building the Model Lifecycle Graph

Back to Devoured - May 07, 2026

DEVOURED

Democratizing Machine Learning at Netflix: Building the Model Lifecycle Graph

Data aiinfrastructuremlmetadata

Netflix built a company-wide graph database that unifies every ML asset—models, features, pipelines, datasets, experiments—into a queryable system using Datomic, solving the discovery crisis that emerges when hundreds of teams build ML independently.

What: Netflix's Model Lifecycle Graph is a centralized Metadata Service that ingests real-time events from ML workflows, normalizes them with a URI-based model, and stores relationships in Datomic and Elasticsearch to enable discovery, lineage tracking, and impact analysis across all teams.

Why it matters: This reveals that at hyperscale, ML infrastructure must solve discovery and reuse before performance—when hundreds of teams build models independently, finding existing work becomes more valuable than building faster.

Deep dive

Netflix built Model Lifecycle Graph (MLG), a centralized Metadata Service that connects fragmented ML assets (models, features, pipelines, datasets, experiments) across the entire company into a single queryable graph
The system ingests real-time events from ML workflows as they happen, capturing asset creation, updates, and relationships between components
Events are normalized using a unified URI-based model that ensures consistent identification of ML assets regardless of which team or platform created them
Storage combines Datomic (immutable, time-aware relationship storage) with Elasticsearch (full-text search and discovery)
Core capabilities include asset discovery (finding existing models and features built by other teams), lineage tracking (understanding dependencies between assets), and impact analysis (identifying downstream effects of changes)
The graph enables cross-domain reuse by making all ML work visible and accessible to every team, reducing duplication and accelerating development
By treating metadata as a first-class concern with real-time ingestion, Netflix ensures the graph stays current as the ML landscape evolves
The URI-based model provides a common language for referring to ML assets regardless of origin, solving the naming and identity problem at scale
This architecture democratizes ML at Netflix by breaking down silos between teams and making the entire corpus of ML work searchable and reusable
The system addresses the discovery bottleneck that emerges when ML infrastructure scales beyond what individual teams can track manually

Decoder

Datomic: Immutable, time-aware database created by Rich Hickey (Clojure creator) that stores data as facts (entity-attribute-value-time tuples), supports querying historical state, and is designed for flexible schemas and complex relational queries.
Metadata Service (MDS): Centralized system that tracks information about data assets (who created them, when, what they contain, how they relate) rather than storing the assets themselves, enabling discovery and governance without data duplication.

Original article

Netflix's Model Lifecycle Graph is a centralized Metadata Service (MDS) that connects fragmented ML assets (models, features, pipelines, datasets, and experiments) across the entire company into a single, queryable graph. By ingesting real-time events, normalizing them with a unified URI-based model, enriching relationships, and storing them in Datomic + Elasticsearch, Netflix enables easy discovery, lineage tracking, impact analysis, and cross-domain reuse of models.

Explore all 73 articles in today's digest