Securing the AI stack from GPU runtime to Kubernetes

Most of the vulnerabilities in an AI application sit below it, in the model, in the GPU runtimes, inference servers, and Kubernetes components it runs on. For example, in one 13-image Kubernetes cluster, those inherited components carried 5,286 CVEs, and replacing them with near-zero CVE images and optimizing them left three, with no application code changes .

Today we're introducing the RapidFort Image Bundle for AI, which applies that approach to the full AI stack. It includes near-zero CVE images for NVIDIA CUDA, AMD ROCm, PyTorch, TensorFlow, Triton, vLLM, and the cluster components underneath, plus PyPI, npm, and Java libraries vetted for malware, tampering, and secrets. You adopt it with a one-line FROM change.

This article walks through each layer of the AI stack, where its CVEs come from, and what the bundle replaces at that layer. It then covers adoption, measured results, and the evidence your security and compliance teams get with every image.

Where CVEs enter the AI stack

Your team writes the model code and application logic, and everything beneath it comes from upstream images and packages. A typical inference deployment stacks a CUDA or ROCm runtime, a framework such as PyTorch, an inference server such as vLLM or Triton, a Python runtime with hundreds of packages, and a Kubernetes cluster running Istio, ArgoCD, Prometheus, and a vector store such as Postgres.

Upstream GPU and ML framework images are large because they're built to serve every use case. A general-purpose CUDA image can include compilers, headers, and libraries that an inference service never loads. Each of those packages is a potential scanner finding, and most findings in an AI workload trace back to base images and dependencies.

That inherited debt lands on your engineers. Scanner findings grow about 140% a year, remediation cycles can run from 60 to 90 days and the average exploitation window is about 12 days. Much of that time goes to triaging CVEs in packages the running workload never touches.

Two constraints make AI stacks harder to clean up than typical web services. Models are pinned to specific CUDA, framework, and runtime versions, so rolling forward to the latest patched image can break training or inference. AI frameworks also pull in thousands of transitive PyPI, npm, and Java dependencies, and coding assistants add packages that no one on the team reviewed.

What the RapidFort Image Bundle for AI covers at each layer

The bundle provides a near-zero CVE replacement for each layer of that stack, built on Ubuntu, Debian, Red Hat, and Alpine. Images are pin-for-pin compatible with their upstream counterparts, so you keep the exact versions your models depend on.

• GPU runtimes: NVIDIA CUDA and AMD ROCm images for training and inference nodes.
• ML frameworks: PyTorch and TensorFlow images, with older and minor versions patched in addition to the latest release.
• Inference servers: NVIDIA Triton and vLLM images for model serving.
• Language runtimes: Python, Node.js, and OpenJDK base images, including FIPS and strict-FIPS variants. The strict Python mode blocks non-approved hashlib fallbacks.
• Open source libraries: Curated Libraries for PyPI, npm, and Maven, checked for malware, tampering, and secrets with third-party threat intelligence.
• Cluster components: minimal images for Kubernetes, Istio, ArgoCD, Prometheus, Grafana, Postgres, and Redis.

RapidFort rebuilds every image daily and patches against published SLAs: vulnerabilities in the CISA Known Exploited Vulnerabilities catalog in one day, Critical and High CVEs in seven days, Moderate in 21, and Low in 28. Images ship for amd64 and arm64. Because the bundle uses standard distributions instead of a vendor-specific OS, you can return to upstream images at any time.

The bundle draws on the RapidFort catalog of tens of thousands of curated near-zero CVE images, which come in many versions, such as minimal, scratch, FIPS, etc. Read more about Curated Images and Curated Libraries.

How to adopt the bundle with a one-line FROM change

For most services, adoption is a change to the base image reference. Point the FROM line at the RapidFort image with the same version tag, rebuild, and run your existing tests.

# Before
FROM vllm/vllm-openai:<version>

# After
FROM <rapidfort-registry>/vllm-openai:<version>

To route Python installs through Curated Libraries, point pip, uv, or Poetry at the curated index, or configure it as an upstream in Artifactory or Nexus.

pip config set global.index-url <curated-libraries-index-url>

Platform teams can mirror the images through pull-through caches in ACR, Artifactory, Google Artifact Registry, Harbor, and Nexus, and use Renovate to keep pinned tags current. Security teams verify each image signature before admitting the image to a cluster.

cosign verify --key <rapidfort-public-key> <rapidfort-registry>/vllm-openai:<version>

Results on a Kubernetes cluster and a Python image

RapidFort applied this process to a 13-image Kubernetes cluster. Swapping to curated images took about one day and eliminated 4,781 of 5,286 CVEs, leaving 505. Profiling and optimizing brought the total to about five days and left three CVEs, eliminating 99.9% of the original findings. No application code was modified.

The same steps on a single Python 3.13 image built on Debian 12 produced these results:

Near-zero CVE means a small number of CVEs can remain, typically where no upstream fix exists yet. Both examples above use general-purpose images.

Across RapidFort customers, teams report about 60% less manual remediation effort and release cycles two to three weeks faster .

Evidence for security and compliance reviews

Every image in the bundle is Cosign signed and carries SLSA provenance that your security team verifies with the RapidFort public key. Each image also ships with an SBOM in CycloneDX JSON or XML, a STIG report, and a CVE report.

For regulated programs, the bundle includes FIPS 140-3 validated images, STIG and CIS benchmarked images hardened to NIST 800-70, and images that are Iron Bank approved and DISA validated.

That per-image evidence shortens the work of assembling a package for FedRAMP, ATO, CMMC, or the EU Cyber Resilience Act. RapidFort customers reach compliance up to 30% faster and take about three months off ATO timelines. Many RapidFort customers have achieved FedRAMP authorization or ATO using the platform and RapidFort Images.

Get started with the RapidFort Image Bundle for AI

The RapidFort Image Bundle for AI secures the software your AI workloads run on. Model behavior, prompt injection, and training data governance need their own controls, and the bundle runs alongside the scanners, CI/CD systems, registries, and cloud platforms you already use.

For developers, adoption is a FROM line and a package index setting. Platform teams get one standard base for every training and inference team. Security and compliance teams get signed provenance and per-image evidence they can hand to auditors.

Schedule a demo to see the RapidFort Image Bundle for AI on your own CUDA, PyTorch, or vLLM images.