Luca BertonZero Trust Was Built for Traditional Workloads — AI Breaks the Model
Zero Trust Architecture (ZTA) assumes "never trust, always verify." But AI workloads challenge this principle in ways traditional applications don't. A model inference endpoint doesn't have a user identity in the traditional sense. Training pipelines require massive data flows that micro-segmentation can throttle. And the "workload" itself — the model — can behave unpredictably in ways that look like compromised behaviour but are actually just stochastic outputs.
This guide maps NIST 800-207 Zero Trust principles to AI-specific infrastructure patterns.
Zero Trust Pillars for AI Infrastructure
Pillar 1: Identity for Non-Human Entities
- Model identity: Every deployed model should have a cryptographic identity (SPIFFE ID, workload identity). This enables mTLS between models, verification of model provenance, and audit trail of which model version served which request.
- Pipeline identity: Training pipelines, data preprocessing jobs, and evaluation runs need workload identities distinct from the humans who authored them. Use service accounts with short-lived credentials.
- API consumer identity: Every system calling your model inference endpoints must authenticate. No anonymous model access — even internal services. This prevents shadow AI usage and enables cost attribution.
- Model signing: Sign model artifacts at build time (Sigstore/cosign). Verify signatures before deployment. A model that can't prove its provenance doesn't get deployed.
Pillar 2: Micro-Segmentation for AI Data Flows
- Training data isolation: Training data storage should be network-segmented from inference endpoints. A compromised inference endpoint should not be able to reach training data (prevents data exfiltration via model manipulation).
- GPU cluster segmentation: Training GPU clusters in a separate network segment from inference GPU clusters. Different security postures: training needs large data inflows; inference needs minimal data access.
- Feature store access control: Feature stores bridge training and inference. Implement column-level access control — inference endpoints only access features they need, not the full feature catalog.
- Model registry as trust boundary: The model registry is your trust boundary between development (untrusted) and production (trusted). Models cross this boundary only after validation, signing, and approval.
Pillar 3: Continuous Verification for AI
- Inference request validation: Every inference request verified for: authenticated identity, rate limiting, input schema validation, and prompt injection detection. Reject before the model ever sees suspicious inputs.
- Output monitoring: Continuous verification doesn't stop at the input. Monitor model outputs for anomalies: sudden distribution shifts, unexpected refusal patterns, or outputs that don't match expected format.
- Behavioural baseline: Establish what "normal" model behaviour looks like (latency distribution, token output distribution, topic distribution). Alert on deviations — they may indicate model poisoning or adversarial manipulation.
- Re-verification on drift: When model monitoring detects significant drift, automatically reduce the model's trust level: add human-in-the-loop verification, restrict to lower-risk use cases, or roll back to previous version.
NIST 800-207 Mapping for AI Systems
Key Tenets Applied
- Tenet 1 (All data sources are resources): Training data, model weights, feature stores, and evaluation datasets are ALL resources requiring protection. Not just "the API endpoint."
- Tenet 2 (All communication is secured): mTLS between all AI system components. Encrypted data in transit between storage → training → registry → inference. No plain-text model weight transfers.
- Tenet 3 (Access granted per-session): Training jobs get time-limited access to data. Inference endpoints get time-limited credentials for feature stores. No persistent access tokens.
- Tenet 5 (Monitor and measure integrity): Extends beyond traditional system integrity to MODEL integrity. Continuous monitoring that model behaviour hasn't been tampered with.
- Tenet 7 (Collect information about assets): Complete asset inventory of all AI components: models (versions, lineage), data (classification, retention), compute (GPU utilisation, access logs), and endpoints (traffic patterns, consumers).
📘 Book
Kubernetes Recipes
Practical guide for container orchestration and deployment — hands-on patterns you can use today.
View on Amazon →Implementation Patterns
Kubernetes-Native Zero Trust for AI
- Cilium network policies: L7-aware policies that can inspect and filter traffic to/from model endpoints. Allow only specific gRPC methods, reject unexpected API calls.
- SPIFFE/SPIRE for workload identity: Every pod (model server, training job, data pipeline) gets a cryptographic identity. mTLS enforced cluster-wide.
- OPA/Gatekeeper for admission control: Enforce that AI workloads can only deploy if they have: signed images, approved model versions, required labels, and resource limits set.
- Istio/Linkerd service mesh: Automatic mTLS, traffic observability, and fine-grained authorization policies between AI microservices.
- Falco for runtime detection: Custom Falco rules for AI-specific threats: unexpected network connections from model serving pods, file system access to training data from inference containers, unusual GPU memory patterns.
zero trust
AI security
enterprise
NIST
micro-segmentation
identity
ML security