sdlc.dev

Docker path · What a container is · Topic 2 of 12

Images and layers

Not started

An image is a read-only stack of filesystem layers plus a little metadata. Each Dockerfile instruction that changes files (RUN, COPY, ADD) produces one layer: a tarball of what was added, changed or deleted compared to the layer below. When a container starts, the layers are stacked into one directory tree and a fresh, empty writable layer goes on top for that container alone.

Why it matters: Image size, build speed, pull time and disk use all come down to layers: what is in each one, and which ones two images share.

  1. 1An image is a stack of layers

    Each layer is a tarball of file changes. A small JSON manifest lists the layers in order, plus a config with the default command, user and environment.

  2. 2Addressed by content

    Every layer and manifest is named by the SHA-256 digest of its bytes, so identical layers are stored and downloaded once.

  3. 3Merged into one view

    A union filesystem stacks the read-only layers and adds a thin writable layer per container on top (copy-on-write).

  4. 4An open format

    The OCI image specification defines the layout, so any compliant registry and runtime can use the same image.

1

How it works

The explanation, with examples you can run.

Looking inside

docker history hello:1 --format '{{.Size}}\t{{.CreatedBy}}'# 0B      CMD ["node" "server.js"]# 0B      USER node# 12.3kB  COPY . . # buildkit# 2.72MB  RUN /bin/sh -c npm ci --omit=dev # buildkit# 16.4kB  COPY package.json package-lock.json ./ # buildkit# 8.19kB  WORKDIR /app# 0B      CMD ["node"]# ...     (the node:22-alpine base layers)

Instructions such as CMD, USER and ENV only change the image config, so they add no file layer. docker image inspect --format '{{json .RootFS.Layers}}' lists the layer digests (sha256:1b34...) that make up the filesystem.

Content addressing

Every piece of an image is named by the SHA-256 hash of its bytes: each layer, the config, and the manifest that lists them. Because the name is derived from the content:

  • two images that share a base share those layers on disk and in the registry, and a pull downloads only the layers you do not already have;
  • a digest can be verified after download, so tampering is detectable;
  • the manifest's own digest identifies the whole image exactly, which is what pinning by digest relies on.

The OCI image specification standardises this layout: an image index (one entry per platform), manifests, configs and layers. Docker, containerd, Podman, and all major registries speak it.

The union filesystem and copy-on-write

At run time the storage layer (overlayfs on most Linux hosts) presents the layers as one merged tree. Reads fall through to the highest layer that has the file. The first write to a file from a lower layer copies it up into the container's writable layer, and the change happens there; the image itself is never modified. That is why a hundred containers from one image start instantly and use almost no extra disk.

Two consequences catch people out:

  1. Deleting does not shrink. Removing a file in a later layer only records a whiteout that hides it. The bytes are still in the earlier layer and still shipped.
  2. Container writes are temporary. Everything in the writable layer disappears with docker rm. Writing heavily there is also slower than writing to a volume.

A recent change: the containerd image store

Docker Engine 29.0 made the containerd image store the default for fresh installations (upgraded daemons keep the classic overlay2 driver until you switch); Docker Desktop already used it. Images are now kept by containerd's content store and unpacked by snapshotters. Day to day the commands are the same, but the store can hold multi-platform images and attestations, and docker image ls reports both disk usage and content (download) size:

docker image ls --tree# IMAGE              ID             DISK USAGE   CONTENT SIZE# hello:1            ba5bcd6124ac        237MB         61.6MB
2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.One RUN step downloads a 200 MB archive; the next RUN step deletes it. How does that affect the image size?
  2. 2.Ten images on your machine all start FROM the same base. How many copies of the base layers are stored?
  3. 3.A container writes a log file under /var/log and you then run docker rm. What happens to the file?

0 of 3 right so far. Get all three to finish this topic.

Go deeper