DEVOURED
From S3 to GPU in one copy: Rethinking data loading for ML training (50 minute video)
Vortex, an open-source columnar format, enables streaming training data from S3 to GPUs at up to 60 Gbps with near-zero copies.
What: Staff engineer Onur Satici introduces Vortex, which uses cascading lightweight encodings and layout-aware pruning to eliminate CPU and NVMe bottlenecks in ML training pipelines. The project is under the LF AI & Data Foundation.
Why it matters: Current ML pipelines waste significant time and compute on data movement and decompression; streaming directly from object storage to GPU memory is the next frontier for training efficiency.
Deep dive
- Zero-copy: A technique where data is moved from source to destination without being duplicated in memory.
- Columnar format: Data storage that organizes values by column rather than row, optimized for analytical queries.
- Layout-aware pruning: Skipping entire segments of data based on statistics (like min/max) before loading it into memory.
- SIMD (Single Instruction, Multiple Data): Parallel processing where one instruction operates on multiple data points simultaneously.
- RDMA (Remote Direct Memory Access): A networking technique that allows data transfer between computer memories without involving the operating system or CPU.
Decoder
- Columnar format: Data storage structure organized by columns, significantly faster for analytical processing and aggregation.
- NVMe: A high-speed storage protocol optimized for SSDs, often a bottleneck in heavy data processing.
Original article
Full article content is not available for inline reading.
Explore all 68 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi