DEVOURED
Scaling Zero Copy from 1 Trillion to 120 Trillion Rows with File Federation
Salesforce scaled its 'Zero Copy' architecture to 120 trillion rows by moving from query federation to Apache Iceberg-based file federation.
What: Salesforce Data 360 shifted its architecture to allow AI workloads to interact with enterprise data across distributed systems without centralizing it. The move to Iceberg file federation aims to reduce compute overhead and maintain strict data governance via temporary catalog access.
Why it matters: Enterprises are increasingly rejecting the 'centralized data lake' model, opting for federated architectures that satisfy AI needs for real-time, cross-platform data access without expensive egress or synchronization.
Decoder
- File Federation: A technique where compute engines access raw data files directly from remote sources using a common metadata format (like Apache Iceberg) rather than routing queries through a database management layer.
Original article
Zero Copy at Salesforce Data 360 evolved from Query Federation to Iceberg File Federation to support AI workloads across distributed enterprise data without centralizing it. The new architecture reduces cross-system compute overhead, preserves governance through temporary catalog-based access, and is being pushed by the need for real-time AI across major data platforms.
Explore all 55 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi