How Data 360 Segmentation Processes a Quadrillion Records Across Arbitrary Customer Data Models

Back to Devoured - June 18, 2026

DEVOURED

How Data 360 Segmentation Processes a Quadrillion Records Across Arbitrary Customer Data Models

Data enterpriseinfrastructure Salesforce

Salesforce engineers are tackling metadata bottlenecks as they scale Data 360 segmentation to process a quadrillion records per month.

What: The system processes Spark jobs for arbitrary customer schemas, with metadata bloat—sometimes reaching 500MB payloads—identified as the primary performance constraint when generating query plans.

Why it matters: This illustrates the extreme metadata challenges faced when managing massive multi-tenant data platforms where standard schema-heavy approaches fail.

Original article

Salesforce's Data 360 segmentation processes 1 quadrillion records per month across arbitrary customer schemas, relationship graphs, and storage systems, while running about 3 million Spark jobs monthly. Metadata became the bottleneck, with some environments hitting 3,000 to 6,000 tables, 500+ MB metadata payloads, and billions of candidate query plans.

Explore all 55 articles in today's digest