DEVOURED
Turning Scattered Data Into Queryable Segments at Scale: How Razorpay Built Its Customer Data Platform
Razorpay manages 500 million user profiles by orchestrating Airflow, Spark, and Temporal to transform raw transaction events into queryable audience segments.
What: The platform uses Spark for daily batch computation of segments, while Temporal manages reliable ingestion into DynamoDB, ensuring zero-downtime versioning and privacy-safe lookups.
Why it matters: Managing massive customer profiles requires moving beyond simple database tables to a unified, versioned platform that can reconcile batch-computed insights with high-frequency transactional data.
Decoder
- DAG (Directed Acyclic Graph): In Airflow, a collection of tasks organized to reflect their relationships and dependencies.
- Temporal: A workflow orchestration engine that ensures long-running processes are durable and can recover from failure.
Original article
Razorpay built an in-house Customer Data Platform to turn scattered transaction data across 500M+ user profiles into real-time, queryable audience segments, using Airflow DAGs + Spark for daily segment computation (with reuse and deduplication), Temporal workflows for reliable DynamoDB ingestion with zero-downtime versioning, and privacy-preserving hashed lookups.
Explore all 71 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi