Image vulnerabilities

01The problem

Every image contains hundreds of software packages, and some of them have known security flaws. Scanners such as Trivy compare an image's packages with databases of known vulnerabilities and list every match.

The platform team scanned all four of Tallybook's candidate images. The current one has hundreds of findings. Nobody can fix hundreds of findings, and a pipeline that blocks on all of them would never release anything. The skill is knowing which findings matter and which choice removes most of them at once.

02The concept

Reading a finding

Two sources of findings

• OS packages come from the base image. You fix most of them by choosing a smaller, newer base image and rebuilding regularly.
• Language packages (npm, pip) are your app's dependencies. You fix them by upgrading in your own code.

A practical rule

Block a release on CRITICAL or HIGH findings that have a fix, warn on the rest, and rebuild images regularly so fixes in the base image arrive without anyone asking.

03Example

Load each scan and count findings by severity and source:

import json
from urllib.request import urlopen

import pandas as pd

base = "https://academy.cloudtechanalytics.com/datasets/cicd/"

def load(name):
    with urlopen(base + name) as f:
        return json.load(f)

rows = []
for variant in ["current", "slim", "alpine", "distroless"]:
    scan = load(f"scan-{variant}.json")
    for result in scan["Results"]:
        for v in result["Vulnerabilities"]:
            rows.append({"image": variant, "source": result["Class"], "severity": v["Severity"],
                         "package": v["PkgName"], "fixable": v["FixedVersion"] != ""})
findings = pd.DataFrame(rows)
pd.crosstab([findings["image"], findings["source"]], findings["severity"])[["CRITICAL", "HIGH", "MEDIUM", "LOW"]]

severity              CRITICAL  HIGH  MEDIUM  LOW
image      source
alpine     lang-pkgs         1     2       2    1
           os-pkgs           0     2       5    3
current    lang-pkgs         1     2       2    1
           os-pkgs           5    38     112  241
distroless lang-pkgs         1     2       2    1
           os-pkgs           0     1       3    2
slim       lang-pkgs         1     2       2    1
           os-pkgs           1     8      27   58

The base image accounts for almost all the findings in the current image; moving to a small base removes nearly all of them in one change. The app's own npm findings are identical in every image: no base image can fix them. Now the blocking rule:

must_fix = findings[findings["severity"].isin(["CRITICAL", "HIGH"]) & findings["fixable"]]
print(must_fix.groupby("image").size().rename("blocking findings"))
print()
print(must_fix[(must_fix["image"] == "distroless")][["source", "severity", "package"]].to_string(index=False))

image
alpine         5
current       30
distroless     3
slim           9
Name: blocking findings, dtype: int64

   source severity      package
lang-pkgs CRITICAL jsonwebtoken
lang-pkgs     HIGH        axios
lang-pkgs     HIGH       lodash

Even the smallest image would be blocked, but only by three findings, all in the app's own dependencies: jsonwebtoken (critical: tokens can be forged), axios and lodash. Those are fixed by upgrading three packages in package.json, which is a normal pull request, not a crisis.

04Walkthrough

1. Run the cells. What share of the current image's findings have a fix available?
2. Which OS packages appear most often in the current image's findings? Why might they be in a web app's image at all?
3. Count the blocking findings for each image after the three npm upgrades.
4. Write the scanning policy (the task below).

05Practice

06Check your understanding