DEVOURED
MurrDB (GitHub Repo)
MurrDB is a fast NVMe/S3-backed serving cache optimized for batch reads/writes of large tabular data for ML/AI inference, offering a cheaper, lower-latency alternative to Redis.
What: MurrDB is a RocksDB-based cache designed for ML/AI inference workloads, providing tiered storage (memory, NVMe, S3), native batch operations over columnar data, zero-copy wire protocol for Python (Numpy/Pandas/Polars/Pytorch), and stateless persistence on S3. Benchmarks show it's significantly faster and uses less RAM than Redis for typical ML ranking use cases.
Why it matters: This project addresses a common pain point in ML/AI serving: efficiently retrieving large, tabular feature data without keeping everything in expensive RAM. It indicates a move towards specialized data infrastructure tailored for the unique demands of AI inference.
Takeaway: If you are dealing with large tabular data for AI inference and finding Redis too slow or expensive for batch operations, consider evaluating MurrDB as a specialized serving cache.
Deep dive
- MurrDB operates as a caching layer between batch data pipelines and inference applications, with data tiered across memory, NVMe, and S3.
- It is designed for batch-in, batch-out operations over columnar storage, allowing direct ingestion of Parquet/Arrow files.
- The zero-copy wire protocol allows direct conversion to Numpy, Pandas, Polars, or Pytorch arrays in Python without additional parsing overhead.
- MurrDB is stateless, persisting all data to S3, which allows it to self-bootstrap and avoid data loss on node eviction.
- It is specifically not a general-purpose database, OLTP system, analytics database, or general-purpose cache.
- Benchmarks indicate MurrDB is approximately 3x faster than Redis on packed-blob reads and 12x faster on HSET layouts for ML ranking, while using about 3x less RAM.
- The project is still in early days but is actively developing features like Arrow Flight/gRPC APIs, more data types, and Apache Iceberg support.
Decoder
- NVMe (Non-Volatile Memory Express): A specification for accessing non-volatile storage media attached via a PCI Express (PCIe) bus, offering much higher throughput and lower latency than traditional SATA SSDs.
- RocksDB: An open-source, embeddable persistent key-value store optimized for fast storage and retrieval on modern hardware, developed by Facebook.
- Zero-copy wire protocol: A communication protocol designed to transmit data without requiring the CPU to copy it from one memory location to another, reducing latency and CPU overhead.
- Columnar storage: A database storage method that stores data tables by column rather than by row, optimized for analytical queries where aggregates are calculated over many rows for a subset of columns.
- Parquet/Arrow files: Optimized columnar data formats used for efficient data interchange and storage in big data ecosystems.
Original article
Full article content is not available for inline reading.
Explore all 61 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi