Taming Vulnerability Chaos Across The Multiverse of Kubernetes and Edge

· 3min · Michiel V.

The talk focused on the unique challenges of vulnerability management in distributed edge environments versus traditional datacenters, and how modern tooling and standards can help.

Some Context

  • Datacenters: centralized, stable connectivity, easier to manage.
  • Edge environments: distributed, often with constrained or intermittent connectivity.
  • Key challenge: vulnerability management across heterogeneous systems and environments.

From Virus Scanners to CVE Scanners

  • Security has evolved from traditional virus scanning to scanning for CVEs (Common Vulnerabilities and Exposures).
  • Modern scans occur at three stages:
    1. Build-time (image creation)
    2. Registry scanning (before deployment)
    3. Runtime scanning (live containers)

Demos

Demo 1: Trivy vs Grype

  • A Claude-driven script ran both scanners on common Linux base images.
  • Findings:
    • Trivy struggled on Alpine images.
    • Grype consistently found more vulnerabilities.
  • Python images were also tested — again, Grype found significantly more CVEs.

Demo 2: SBOMs

  • SBOM (Software Bill of Materials) shows exactly which packages and dependencies are in a build.
  • Using Syft, they generated SBOMs, then scanned them with Trivy and Grype.
  • Findings:
    • Some software (especially with licensed dependencies) can obscure or remove evidence of components during the build.
    • Grype again outperformed Trivy in uncovering vulnerabilities.

Demo 3: Removing CVEs via Metadata Manipulation

  • A script modified /var/lib/dpkg/status to remove CVE references.
  • This tricks scanners into thinking vulnerabilities do not exist.
  • Highlights the need for trusted build and provenance verification rather than relying on local metadata alone.

For a view on the concerned code have a peek at Denis' repository.

Securing the Build & Deployment Pipeline

Sigstore

  • Provides digital signatures for container images, ensuring integrity and protecting against metadata manipulation.
  • Should be integrated into admission control policies.
  • Sigstore
  • Cosign
  • Rekor

SLSA Framework

  • Supply-chain Levels for Software Artifacts — defines levels of build assurance.
  • Encourages provenance tracking, reproducible builds, and higher trust.
  • https://slsa.dev/
  1. Secure from source: Utilize hardened base images such as Wolfi or Chainguard to ensure a minimal and secure foundation.
  2. Generate SBOMs during the build process: Employ tools like Syft to create Software Bill of Materials, providing transparency into your application's components.
  3. Build APK packages declaratively: Use Melange to define your application's dependencies and build process in a single YAML file, facilitating reproducible and auditable builds.
  4. Compose images with Apko: Leverage Apko to create single-layer container images based on the APK package format, integrating the APKs built with Melange into your container images.
  5. Ensure full provenance and signing: Implement tools like Sigstore to sign your images and maintain a transparent record of your build process.
  6. Aim for SLSA Level 3: Strive to achieve SLSA Level 3 to ensure the highest level of build and supply chain integrity.

True security begins at the source, not just at the container runtime.