Painless Docker - 2nd Edition

Optimizing Docker Builds: Caching Strategies & Best Practices

As a reminder, each instruction in a Dockerfile creates a new layer in the image. Docker uses a build cache to speed up the build process by reusing layers that haven't changed. Docker understands that a layer hasn't changed if the instruction and its context (the files and environment used to execute the instruction) are the same as in a previous build. Docker can reuse the cached layer instead of re-executing the instruction.

# A simple example
RUN apt install -y package1 package2

If you rebuild the image:

• The instruction is the same
• The context of the instruction is the same (i.e., the installed packages haven't changed, the base image is the same, etc.)

Then Docker can reuse the cached layer for this instruction.

Let's take a more advanced example: a Python application with dependencies specified in a requirements.txt file. If you're not familiar with Python, requirements.txt is a common way to list the packages your application depends on. It has this format:

flask==2.0.1
requests==2.25.1
numpy==1.21.0

Here, flask, requests, and numpy are the package names, and the numbers after == specify the exact versions to install. To install these packages, you would typically need pip (the Python package installer) or a similar tool, and you would run the following command:

pip install -r requirements.txt

Consider the following Dockerfile for a Python application:

# Use the official Python base image
FROM python:3.14-slim
# Set the working directory
WORKDIR /app
# Copy the application code
COPY . .
# Install the dependencies
RUN pip install -r requirements.txt
# Run the application
CMD ["python", "app.py"]

Note that the requirements.txt file is copied along with the rest of the application code using the COPY . . instruction. So COPY . . copies the application code and the requirements.txt file.

The instruction RUN pip install -r requirements.txt can take a long time to execute, especially if there are many Python packages to install. However, if the requirements.txt file hasn't changed since the last build, Docker can reuse the cached layer for this instruction - as a result, the build process will be much faster.

However, once an instruction preceding the RUN pip install -r requirements.txt instruction changes, Docker will invalidate the cache for all subsequent layers, including the RUN pip install -r requirements.txt instruction.

For example, you modified your application code as part of a regular update. This means that the COPY . . instruction has changed because the application code is different. As a result, Docker will invalidate the cache for the COPY . . instruction and all subsequent instructions, including the RUN pip install -r requirements.txt instruction.

As a result, even if the requirements.txt file hasn't changed, Docker will have to re-execute the pip install command because the previous layer has changed, which is a long-running operation - this is inefficient.

The golden rule here is that you need to take this into consideration when ordering the instructions in your Dockerfile.
In practice, place frequently changing instructions (like copying application code) after less frequently changing instructions (like installing dependencies) to maximize cache efficiency. To better understand this concept, consider the following modified Dockerfile:

# Use the official Python base image
FROM python:3.14-slim
# Set the working directory
WORKDIR /app
# Copy the requirements file
COPY requirements.txt .
# Install the dependencies
RUN apt-get update && \