DEVOURED
Diving deep into Redis's new array data type
Redis 8.8 introduces a new native Array data type, designed by Salvatore Sanfilippo, providing constant-time positional access for sparse or dense arrays where index itself carries semantic meaning.
What: Redis 8.8 introduces the Array data type, designed by Salvatore Sanfilippo, addressing the need for constant-time positional access. Unlike lists (O(N) access) or hashes (no range queries), Arrays efficiently handle dense and extremely sparse data by dividing the index space into 4096-slot groups, only allocating memory for groups with data. Commands include `ARGETRANGE`, `ARSCAN`, `ARCOUNT`, `ARLEN`, `ARRING` (for fixed-size ring buffers), `ARGREP` (server-side pattern matching with regex), `AROP` (server-side aggregations like SUM, MIN, MAX), and `ARDEL`/`ARDELRANGE`.
Why it matters: This new data type addresses a critical gap in Redis's offerings for use cases where the numerical index itself is semantically important, pushing Redis further into domains previously requiring complex workarounds or external databases, while maintaining its core strengths of speed and memory efficiency.
Takeaway: If you're using Redis and managing data where numeric indices have intrinsic meaning (e.g., log line numbers, port IDs, event sequences), explore the new Array type in Redis 8.8 for more efficient and idiomatic solutions compared to lists or hashes.
Deep dive
- The new Redis Array type, introduced in Redis 8.8, provides effectively constant-time (O(1)) positional access by index, filling a gap where other data types like lists (O(N) for index access) or hashes (no range queries) fall short.
- It is designed for scenarios where the index itself carries semantic meaning, such as document line indexing, stack trace analysis, or workflow steps.
- The internal implementation divides the index space into groups of 4096 slots, only allocating memory for groups that contain data, making it highly efficient for sparse arrays. Empty groups cost only 8 bytes.
- Small integers, floats, and short strings are stored directly inside pointer slots, optimizing memory usage for dense arrays of small values.
- For very large arrays (indices >= 8,388,608), the structure silently promotes to a three-level hierarchy (superdir -> block -> group) to maintain O(1) access without unbounded directory growth.
- Key commands include:
ARGETRANGE: Returns a range of values, including empty slots.ARSCAN: Iterates only over occupied slots in a range, skipping gaps.ARCOUNT: Returns the total count of active entries in O(1).ARLEN: Returns the highest occupied index + 1, indicating data extent.ARRING: Manages a fixed-size ring buffer for event logs atomically.ARGREP: Performs server-side pattern matching (EXACT, MATCH, GLOB, RE) on occupied slots, skipping empty ones.AROP: Executes server-side aggregations (SUM, MIN, MAX, AND, OR, XOR, USED, MATCH) over a specified range.ARDEL,ARDELRANGE: Deletes single entries or ranges, freeing memory for emptied slices immediately.- The article contrasts Arrays with Lists, Hashes, Sets, and Sorted Sets, highlighting that Arrays are superior when the numeric index is part of the data model's meaning, not just an implementation detail.
Decoder
- Redis Array: A new native data type in Redis 8.8 designed for efficient storage and retrieval of data where the numerical index is semantically important, offering constant-time access and optimized handling of sparse data.
- Constant-time positional access (O(1)): The ability to retrieve an element at a specific index in a data structure with a fixed amount of time, regardless of the size of the data structure.
- Sparse array: An array where most of the elements have a default or null value, and only a small fraction of elements hold significant data. The Redis Array optimizes storage by not allocating memory for empty slots.
- Ring buffer: A fixed-size circular data buffer where new data overwrites the oldest data when the buffer is full, often used for event logs or streaming data.
Original article
Full article content is not available for inline reading.
Explore all 55 articles in today's digest
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi