sdlc.dev

Redis path · Why it is fast · Topic 2 of 11

Data structures, and choosing one

Not started

Redis is often described as a key-value store, but the values are data structures with their own operations. The value under a key can be a string, a hash of fields, a list, a set, a sorted set or a stream (and more specialised types such as bitmaps, HyperLogLogs, geospatial indexes and, from Redis 8, JSON and vector sets). The commands for each type run on the server, atomically, so the right choice often removes a race and a round trip at once.

Why it matters: Picking the structure that matches the question turns a read-modify-write dance into one atomic server-side command.

  1. 1Values have types

    A key holds a string, hash, list, set, sorted set or stream (among others), and each type has its own commands.

  2. 2Let the server do the work

    INCR, HINCRBY, SADD and ZINCRBY change data in place, atomically, instead of reading it out and writing it back.

  3. 3Sorted sets rank

    Members ordered by a score give leaderboards, priority queues, time-ordered indexes and sliding windows.

  4. 4Small collections are compact

    Small hashes, lists, sets and sorted sets are stored in a compact listpack encoding, which saves a lot of memory.

1

How it works

The explanation, with examples you can run.

The core types

All of these outputs come from redis-cli:

# String: bytes up to 512 MB; also counters> SET user:42:name AnaOK> INCR page:home:views(integer) 1> INCRBY page:home:views 10(integer) 11 # Hash: fields under one key> HSET user:42 name Ana country DK plan pro(integer) 3> HINCRBY user:42 logins 1(integer) 1 # List: ordered, push and pop at both ends> LPUSH jobs:email j1 j2 j3(integer) 3> RPOP jobs:email"j1" # Set: unique members> SADD post:7:likes u1 u2 u3 u2(integer) 3> SISMEMBER post:7:likes u9(integer) 0 # Sorted set: members ordered by a score> ZADD leaderboard 3100 ana 2900 ben 3350 cy(integer) 3> ZINCRBY leaderboard 300 ben"3200"> ZRANGE leaderboard 0 -1 REV WITHSCORES1) "cy"2) "3350"3) "ben"4) "3200"5) "ana"6) "3100"

SADD counted three, not four: u2 was already there. Streams, an append-only log with consumer groups, have their own topic later.

Choosing

You wantUseCommands
A cached blob, a flag, a counterStringGET, SET, INCR, SET ... NX
An object whose fields change separatelyHashHSET, HGET, HINCRBY
A simple queue or a capped recent-items listListLPUSH, RPOP, BLMOVE, LTRIM
Uniqueness and membershipSetSADD, SISMEMBER, SINTER
Ranking, by score or by timeSorted setZADD, ZINCRBY, ZRANGE, ZREMRANGEBYSCORE
An event log with acknowledgementsStreamXADD, XREADGROUP, XACK

Some patterns: a sorted set with timestamps as scores is a time-ordered index (ZRANGE ... BYSCORE for a time window) or a sliding-window rate limiter. LPUSH plus LTRIM 0 99 keeps the latest hundred items. A set per tag plus SINTER answers "posts with both tags".

Encodings and size

Redis picks an internal encoding per value. Small hashes, lists, sets and sorted sets use a listpack, a compact contiguous block; above a size threshold (configurable, for example hash-max-listpack-entries) they convert to a hash table or skip list, which is faster for large sizes and uses more memory. Strings holding integers are stored as integers.

> OBJECT ENCODING user:42"listpack"> OBJECT ENCODING page:home:views"int"> MEMORY USAGE user:42(integer) 84

This is why many small hashes can take less memory than the same data as separate string keys: each top-level key has its own overhead.

Naming keys

Keys are binary-safe strings; the convention is colon-separated segments, object:id:field, such as user:42:sessions. Keep them readable but not huge, and never rely on listing them with KEYS in production. If you need to find all keys for a user, keep that index yourself, for example in a set.

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.You need a live top-10 of players by score, updated thousands of times a second. Which structure fits?
  2. 2.A user object has 20 fields and the app often updates just the last-login time. What is the usual choice?
  3. 3.Which structure answers "has this user already liked this post?" directly?

0 of 3 right so far. Get all three to finish this topic.

Go deeper