sdlc.dev

MongoDB path · Scale and change · Topic 12 of 12

WiredTiger: cache, checkpoints, journal

Not started

WiredTiger has been MongoDB's default storage engine since 3.2, and the only one in recent releases apart from the in-memory engine of Enterprise. It stores each collection and each index in its own file as a B-tree, keeps hot pages in an internal cache, and makes changes durable with a combination of a write-ahead journal and periodic checkpoints. You rarely configure it, but its behaviour explains most of what you see in production.

Why it matters: Most MongoDB performance problems end up as cache problems: a working set that no longer fits, or too many writes waiting to be flushed. Knowing what the storage engine does tells you what to measure.

  1. 1An internal cache in front of the files

    By default the larger of 50% of (RAM minus 1 GB) or 256 MB. The filesystem cache holds compressed file pages on top of that.

  2. 2Checkpoints every 60 seconds

    WiredTiger writes a consistent snapshot of the data files about once a minute; the last checkpoint is always a valid recovery point.

  3. 3The journal covers the gap

    Changes since the last checkpoint are in the write-ahead journal, flushed every 100 ms or immediately for j true and majority writes.

  4. 4Documents are locked, not collections

    WiredTiger uses document-level concurrency with MVCC snapshots, so writers to different documents do not block each other.

1

How it works

The explanation, with examples you can run.

The cache

The internal cache holds uncompressed pages of collections and indexes. Its default size is the larger of 50% of (RAM minus 1 GB) and 256 MB. On the test container, which saw 7.8 GB of memory:

db.serverStatus().wiredTiger.cache["maximum bytes configured"]   // 3570401280, about 3.3 GiB

That is half of (7,835 MB minus 1,024 MB). The operating system's filesystem cache keeps compressed file pages as well, so leave memory for it rather than raising the WiredTiger cache to nearly all of RAM. The quantity to watch is the working set: the indexes and documents your queries touch regularly. While it fits in cache, reads are memory reads. When it stops fitting, reads go to disk, and eviction (writing changed pages out to make room) starts to compete with your workload; the wiredTiger.cache section of serverStatus shows bytes in cache, dirty bytes and pages read into cache.

Checkpoints and the journal

WiredTiger gives each operation a snapshot of the data (multiversion concurrency control) and, about every 60 seconds, writes a consistent snapshot of all data files to disk: a checkpoint. The previous checkpoint stays valid until the new one is complete, so the files on disk are always consistent as of some checkpoint.

Changes made since then are recorded in the journal, a write-ahead log. The journal is flushed to disk every 100 ms, and at once for writes with j: true or with w: "majority" (which implies journaling by default). After a crash, mongod opens the last checkpoint and replays the journal. A w: 1 write without j: true that was acknowledged but not yet flushed can therefore be lost if the machine fails, which is one more reason for majority writes.

Concurrency and compression

Writes take document-level concurrency control: two clients updating different documents of one collection proceed in parallel, and a conflict on the same document is retried internally (or surfaces as a WriteConflict in a transaction). Some operations, such as dropping a collection, still take stronger locks.

Collection data is compressed with snappy by default (zstd for time series collections), indexes with prefix compression, and the journal with snappy. On the test data:

const s = db.orders.stats()s.wiredTiger.creationString.match(/block_compressor=\w+/)[0]   // block_compressor=snappys.size          // 27678888, documents uncompresseds.storageSize   // 8839168, on disk

You can choose zstd or zlib per collection when you create it, trading CPU for disk. The compression applies on disk only; in the cache, pages are uncompressed, so the cache needs room for the data as your queries see it.

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.mongod crashes 40 seconds after the last checkpoint. How does it recover the writes made since?
  2. 2.A server has 32 GB of RAM and default settings. Roughly how large is the WiredTiger internal cache?
  3. 3.Collection stats show size 27.7 MB and storageSize 8.8 MB. Why the difference?

0 of 3 right so far. Get all three to finish this topic.

Go deeper