Shrink, Scan, Confine: A Container Hardening Playbook from Base Image to Unikernel

01The attack surface lives in the base image

Most of the code in a typical container image was not written by the team that ships it. The base image contributes the C library, TLS stack, shell, coreutils and a package manager — and every one of those packages is a CVE the scanner will report and a tool an attacker can use after a foothold. Shrinking the base image removes both problems at once: fewer packages to patch, and fewer utilities available for post-exploitation.

Choosing a base image: a comparison

The base image fixes the floor of your attack surface. The main minimalist options differ in size, default identity, debuggability and support model:

02Distroless and the price of no shell

Distroless images are the strongest general-purpose runtime base, but removing the shell and package manager moves work elsewhere. Four problems come up in every adoption; each has a practical answer.

# Ephemeral debug container sharing the app's process namespace
kubectl debug -it pod/web-7d4c9 \
  --image=busybox:1.36 \
  --target=app \
  -- sh

# Inside: inspect the app process without a shell in its own image
ls -l /proc/1/root/app
cat /proc/1/status | grep -E 'Uid|CapEff' 

03Minify what you already ship: slim and unbase_oci

Rebuilding every image on a new base takes time. Two tools shrink existing images without changing the application, using opposite strategies.

slim: dynamic analysis

slimtoolkit/slim runs the container, observes which files and libraries the application actually touches, and builds a new image containing only those. The project reports reductions of up to 30x, more for compiled languages. The trade-off is coverage: code paths not exercised during the probe are removed, so drive it with realistic probes and pin anything loaded lazily.

# Run the image, probe its HTTP endpoints, keep only what was used
slim build \
  --target registry.example/web:1.8 \
  --tag registry.example/web:1.8-slim \
  --http-probe=true \
  --include-path /app/templates   # keep lazily loaded files explicitly

docker image ls registry.example/web

unbase_oci: static analysis against the base

unbase_oci from the Garden Linux project needs no running container. It compares the application image with the base image it was built on, keeps every file the application layers added or modified, resolves their dynamic library dependencies, and prunes the rest of the base. A wrapper script moves images in and out of Podman as OCI archives and runs the analysis in its own container:

# -l auto: detect the base image automatically from the image's layers
./unbase_oci -l auto podman:demo :unbased

podman image ls | grep demo

Measure the result layer by layer with dive, which shows what each layer adds and how much space is wasted:

# Fail the pipeline if the image wastes space across layers
dive --ci \
  --lowestEfficiency=0.95 \
  --highestUserWastedPercent=0.10 \
  registry.example/web:1.8-slim

04A supply chain you can verify

A hardened image only stays hardened if the pipeline that produces it is controlled. Three practices make the chain verifiable end to end.

# Sign by digest, never by tag
cosign sign --key cosign.key registry.example/web@sha256:3f1c9e0b7a2d4c6e8f00112233445566778899aabbccddeeff00112233445566

# Verify before deploy (also enforced at admission below)
cosign verify --key cosign.pub registry.example/web@sha256:3f1c9e0b7a2d4c6e8f00112233445566778899aabbccddeeff00112233445566

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: verify-signed-images
spec:
  validationFailureAction: Enforce      # fail closed
  webhookTimeoutSeconds: 30
  rules:
    - name: require-cosign-signature
      match:
        any: [{ resources: { kinds: ["Pod"] } }]
      verifyImages:
        - imageReferences: ["registry.example/*"]
          mutateDigest: true             # pin the verified digest into the Pod
          attestors:
            - entries:
                - keys:
                    publicKeys: |-
                      -----BEGIN PUBLIC KEY-----
                      <cosign.pub contents>
                      -----END PUBLIC KEY-----

The secure Dockerfile checklist

Harden the image in the Dockerfile itself. Every line below maps to one rule: run as a non-root user, pin every version, clean caches in the same layer that created them, and prefer COPY over ADD, whose remote-URL download and automatic archive extraction make the build less transparent.

# Pin the base: never :latest
FROM python:3.11-slim-bookworm

# Create a system user and group for the app
RUN groupadd --system app && useradd --system --gid app --no-create-home app

# Install OS packages and clean the cache in the SAME layer
RUN apt-get update \
 && apt-get install -y --no-install-recommends libpq5=15.* \
 && apt-get clean \
 && rm -rf /var/lib/apt/lists/*

WORKDIR /app
# Pin application dependencies exactly (fastapi==0.104.1 in requirements.txt)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# COPY, not ADD; hand ownership to the app user
COPY --chown=app:app . .

# Drop root before the process starts
USER app
CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

05Why scanners disagree — and which findings are noise

Run three scanners against the same image and you get three different answers. Chainguard’s analysis of common base images makes the spread concrete:

Source: Chainguard analysis of base images. The tags are dated (node:17, debian:11.2); read the table for the spread between scanners, not for current counts.

The same node:17 image reports 122, 112 or 26 high-severity findings depending on the tool. Most of that gap is not missed vulnerabilities — it is disagreement about which findings are real. Four sources of false positives account for most of it:

Kernel CVEs belong to the node

Because kernel exposure is a node property, the node operating system deserves the same minimisation as the image. Scanning several node OS images with the same tool shows the scale of the difference:

Results from sudo grype dir:/ on each OS. Talos Linux has no shell, so its CVEs were gathered from its SBOM files as described in the Talos documentation. Source: the Talos Linux project’s published comparison.

img="registry.example/web:1.8-slim"

# Distro-aware scan of the final image only; fail on fixable high/critical
grype "$img" --only-fixed --fail-on high

# Apply recorded not_affected decisions (OpenVEX) so reviewed noise stays suppressed
trivy image --vex ./vex/web.openvex.json --severity HIGH,CRITICAL --exit-code 1 "$img"

# Node OS: scan the host filesystem, not the image, for kernel and OS packages
sudo grype dir:/ --only-fixed

06Confine the running container

A secure image is necessary but not sufficient. The runtime decides what a compromised process can do.

Never privileged; drop every capability you do not need

A container with securityContext.privileged: true is effectively root on the host and bypasses kernel permission checks. The flag bundles the container with the host rather than isolating it; it exists for administrative workloads that manipulate the network stack or hardware. Avoid it and grant specific capabilities instead.

Linux splits root’s privileges into distinct capabilities that can be enabled and disabled independently. Contrary to what most teams expect, leaving securityContext.capabilities empty does not mean “no capabilities”: the runtime applies its default set, which is far more generous than most processes need. A simple web application does not need CAP_SYS_ADMIN (which allows mounting filesystems) or CAP_NET_ADMIN (which allows network configuration changes). Drop everything and add back only what the application requires — typically CAP_NET_BIND_SERVICE to bind a port below 1024 — and a compromise yields far less.

Finding the minimal set is an allow-list exercise: start with no capabilities and add them one use case at a time. Running under SELinux in permissive mode and reading the audit log, or tracing capability checks while the test suite runs, shows exactly which capabilities the application exercises.

apiVersion: v1
kind: Pod
metadata:
  name: web-app
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 10001
    seccompProfile:
      type: RuntimeDefault          # filters the most dangerous syscalls
  containers:
    - name: app
      image: registry.example/web:1.8-slim
      securityContext:
        privileged: false
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true  # nothing can be written to the image filesystem
        capabilities:
          drop: ["ALL"]               # remove the runtime's default set
          add: ["NET_BIND_SERVICE"]   # add back only what the app needs
      volumeMounts:
        - name: tmp
          mountPath: /tmp             # writable scratch space on a volume, not the rootfs
  volumes:
    - name: tmp
      emptyDir: {}

Secrets from a secrets manager, not from the image

The Container Storage Interface (CSI) is the Kubernetes API for exposing storage systems to containers; it lets third-party providers plug new storage types in as volumes. The Secrets Store CSI Driver, developed and maintained by the Kubernetes community, uses that interface to read secrets from a centralised secrets management system (SMS) — HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, Google Secret Manager — and mount them into the Pod as a regular volume. The secret never exists in the image and need not exist as a Kubernetes Secret object at all.

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: web-db-creds
spec:
  provider: vault
  parameters:
    vaultAddress: "https://vault.internal:8200"
    roleName: "web"                         # Vault Kubernetes-auth role
    objects: |
      - objectName: "db-password"
        secretPath: "secret/data/web/db"
        secretKey: "password"
---
# In the Pod spec
volumes:
  - name: secrets
    csi:
      driver: secrets-store.csi.k8s.io
      readOnly: true
      volumeAttributes:
        secretProviderClass: web-db-creds
# ...and in the container
volumeMounts:
  - name: secrets
    mountPath: /mnt/secrets                 # the app reads /mnt/secrets/db-password
    readOnly: true

Least privilege on the network, too

Capabilities limit what a process can do on its node; a service mesh limits whom it can talk to. With Istio, an AuthorizationPolicy expresses exactly which callers may reach which endpoints. The classic example lets the monitoring stack scrape metrics and nothing else:

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: prometheus-scraper
  namespace: istio-system       # root namespace: applies to matching Pods in all namespaces
spec:
  selector:
    matchLabels:
      has-metrics: "true"       # only Pods that expose metrics
  action: ALLOW
  rules:
    - from:
        - source:
            namespaces: ["prometheus"]   # callers from the prometheus namespace...
      to:
        - operation:
            methods: ["GET"]             # ...may only GET
            paths: ["/metrics/*"]        # ...the metrics endpoints

Adapted from Kubernetes Patterns, 2nd ed., Example 24-6. security.istio.io/v1 is the current API; older material uses v1beta1. Remember that an ALLOW policy denies all other traffic to the selected workloads, so add a policy for application traffic before rolling this out.

When a process boundary is not enough: unikernels with urunc

urunc is a specialised OCI runtime, similar to runc, designed to run unikernels instead of standard Linux containers. A unikernel is a single-purpose machine image built by compiling the application together with only the bare minimum of operating-system libraries it needs. The result is tiny, fast to boot, and isolated by a hypervisor rather than by kernel namespaces: urunc starts and manages each unikernel as a lightweight virtual machine, using monitors such as QEMU/KVM, Firecracker or Solo5.

This changes the threat model in section 08: a kernel exploit inside a unikernel attacks the unikernel’s own library OS, not the shared host kernel. The cost is that the workload must be built as a unikernel, which suits focused network services far better than general applications.

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: urunc
handler: urunc              # containerd runtime handler configured on the node
---
apiVersion: v1
kind: Pod
metadata:
  name: edge-proxy
spec:
  runtimeClassName: urunc
  containers:
    - name: proxy
      image: registry.example/edge-proxy-unikernel:2.1

07Triaging runtime alerts

Runtime detection fires constantly; the skill is separating an attack from routine operations. For each alert, ask the context question first, then look for the signals that make it a true positive:

Two of these rules are cheap to deploy and rarely noisy in production. The Falco rule catches a language runtime spawning a shell; the audit query lists every exec for comparison against change tickets.

- rule: Language Runtime Spawned Shell
  desc: A Python, Java, Node or PHP process started a shell inside a container
  condition: >
    spawned_process and container
    and proc.name in (sh, bash, dash, ash)
    and proc.pname in (python, python3, java, node, php, php-fpm)
  output: >
    Shell spawned by language runtime
    (parent=%proc.pname cmd=%proc.cmdline container=%container.name
     image=%container.image.repository)
  priority: CRITICAL
  tags: [container, shell, mitre_execution, T1059]

# Requires an audit policy that records pods/exec at Metadata level or higher
jq -c 'select(.objectRef.subresource == "exec")
  | {ts: .requestReceivedTimestamp, user: .user.username, ip: .sourceIPs,
     ns: .objectRef.namespace, pod: .objectRef.name}' \
  /var/log/kubernetes/audit/audit.log

08The threats this playbook is built for

Three widely exploited vulnerabilities show why every layer above matters — and, just as important, where each one is fixed. Rebuilding an image fixes none of the last two.

Use the checklist as the release gate for a workload. It scores itself; anything short of full coverage is an open item.

Sources

• GoogleContainerTools/distroless
• slimtoolkit/slim
• gardenlinux/unbase_oci
• wagoodman/dive
• nubificus/urunc
• Secrets Store CSI Driver documentation
• Istio AuthorizationPolicy reference
• Kubernetes: configure a security context
• Sigstore Cosign documentation
• CVE-2024-2961 · CVE-2024-21626 · CVE-2024-1086
• Bilgin Ibryam and Roland Huß, Kubernetes Patterns, 2nd ed. (O’Reilly): process containment, network segmentation and secure configuration chapters
• Related: Docker’s layered model — secrets, multi-stage builds and capabilities