DEVOURED
TimeSeries Tiered Storage Journey: Kafka/Flink Streams to Native Cassandra Cold Reads
Netflix reduced operational costs by shifting cold TimeSeries data reads directly from S3 backups instead of maintaining a dedicated Flink/Kafka compaction pipeline.
What: Netflix stores multi-petabyte temporal data by tiering older segments from Cassandra to S3. Their new native cold-tier architecture eliminates the streaming/compaction layer, improving p90 latency by 30% while serving 15+ PB of data.
Why it matters: This highlights a trend of bypassing middleman streaming layers in favor of direct object store access for historical data to simplify infrastructure and reduce latency.
Original article
Netflix's TimeSeries Abstraction stores and queries multi-petabyte temporal datasets, tiering older, immutable slices from hot Cassandra into cheaper S3 storage. The earlier design used Kafka and Flink to stream Parquet to S3 and daily compaction, serving cold reads with roughly 500 ms p99 latency, but added operational cost. The later Cassandra-native cold tier reads directly from S3 backups, serves 15+ PB of compressed cold data, and improves p90 latency by about 30% while preserving the same query API.
Explore all 58 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi