Curious Pandas

Layers and the build cache

Every Dockerfile step is a cached layer, and one change rebuilds everything after it. Reorder two lines and stop reinstalling the internet on every code change.

14 min+35 XP for finishing

An image is built one Dockerfile instruction at a time, and each step’s result is kept as a layer. Build again and Docker checks each step against its cache: if the step and everything it depends on are unchanged, it reuses the old layer instead of running the step.

Two rules decide whether a step is a cache hit. Both come straight from Docker’s documentation:

  • RUN is matched on its command text alone. The builder doesn’t look at what the command would download. Same text, cache hit.
  • COPY is matched on a checksum of the files it copies. Edit one of those files and the checksum changes.

And the rule that makes ordering matter: once one step misses, every step after it is rebuilt. In the docs’ own words, “Once the cache is invalidated, all subsequent Dockerfile commands generate new images and the cache isn’t used.”

Watch the cascade

This project is a small Node.js app. The first Dockerfile copies everything, then installs dependencies. Edit src/server.js and watch which steps rebuild. Then switch to “Package files first” and do the same.

Build cache simulatorpretend timings
Dockerfile:
Change something, then the build runs again

    With everything copied first, a one-character code change changes the checksum of COPY . .. That invalidates RUN npm ci after it, so you reinstall every dependency on every edit. Copy just package.json and the lockfile first, install, and then copy the rest. Now a code change only misses at the second COPY, and the slow install stays cached. It only reruns when the dependency files themselves change, which is exactly when it should.

    Check yourself+10 XP

    Your Dockerfile has COPY . . on line 3 and RUN npm ci on line 4. You fix a typo in README.md. Does npm ci run again?

    Check yourself+10 XP

    Which line order keeps dependency installs cached across ordinary code changes?

    Check yourself+10 XP

    You add a dependency to package.json. With the cache-friendly order, what rebuilds?

    Do it for real · your machine+50 XP bonus

    Everything above ran in your browser. Now do it on a real cluster withthe tilt platform. You need Docker, kind and Tilt, about 15 GB of free disk and 8 GB of spare RAM. Setup guide · Can my machine run it?

    In any Node project with Docker:

    docker build -t cache-demo .        # first build: everything runs
    docker build -t cache-demo .        # again: every step shows CACHED
    echo "// edit" >> src/server.js
    docker build -t cache-demo .        # which steps rebuilt?
    docker history cache-demo           # one line per layer

    Swap your Dockerfile to the package-files-first order and repeat the edit.

    Self-reported. We can't see your machine, and we don't want to. This is a bonus, never required.

    Sources

    Finished reading?

    Missions and quizzes above earn their own XP. This marks the lesson done.