Luca BertonEncryption protects data at rest (stored) and in transit (moving). But during processing — when AI models actually use the data — it's traditionally unprotected in memory. Confidential computing changes this by protecting data in use through hardware-based trusted execution environments (TEEs).
For AI workloads processing sensitive data — patient records, financial transactions, classified information — confidential computing closes the last gap in data protection.
The Data Protection Gap
Traditional security model:
- At rest: Encrypted storage (AES-256, customer-managed keys) ✓
- In transit: TLS encryption for all data movement ✓
- In use: Data decrypted in memory during processing. Accessible to hypervisor, cloud provider administrators, and anyone with memory access. ✗
This means that during AI inference, your sensitive data is exposed in memory — potentially accessible to the cloud provider's infrastructure stack, compromised hypervisors, or insider threats.
How Confidential Computing Works
Trusted Execution Environments (TEEs)
- Hardware isolation: CPU creates an encrypted, isolated memory region (enclave) that even the operating system and hypervisor cannot access
- Memory encryption: Data in the enclave is encrypted in memory. Even physical memory access (cold boot attacks, bus snooping) reveals only encrypted data.
- Attestation: Cryptographic proof that the enclave is running the expected code on genuine hardware — verifiable by the data owner before sending sensitive data
Hardware Options
- Intel SGX (Software Guard Extensions): Process-level enclaves. Limited memory (256MB-1GB). Best for small, sensitive computations.
- Intel TDX (Trust Domain Extensions): VM-level isolation. Full VM runs in a trusted domain. Better for AI workloads that need more memory.
- AMD SEV-SNP: VM-level encryption with strong isolation. Available on Azure, GCP, and AWS.
- ARM CCA (Confidential Compute Architecture): Realm-based isolation for ARM processors. Relevant for edge AI.
- NVIDIA H100 Confidential Computing: GPU TEE capability for AI inference and training. The most relevant development for AI workloads.
Kubernetes Recipes
A practical guide for container orchestration and deployment by Grzegorz Stencel & Luca Berton (Apress).
Watch on Skillshare →Confidential AI Use Cases
Where Confidential Computing Adds Value
- Multi-party AI: Multiple organizations contribute data to train a shared model without exposing their data to each other or the platform operator
- Cloud AI with sovereignty: Process regulated data in public cloud infrastructure while maintaining cryptographic proof that the cloud provider cannot access the data
- Model IP protection: Protect proprietary model weights during inference — the model runs in an enclave that prevents extraction
- Healthcare AI: Process patient data for AI inference without exposing it to cloud infrastructure operators
- Financial AI: Run AI models on sensitive financial data with hardware-backed guarantees that meet regulatory requirements
Challenges and Limitations
- Performance overhead: Confidential computing adds 5-30% overhead depending on workload and TEE technology. GPU confidential computing has lower overhead than CPU-based approaches.
- Memory limitations: SGX enclaves have limited memory. TDX/SEV-SNP support larger workloads but still constrain some large model deployments.
- Side-channel risks: TEEs protect against direct memory access but some side-channel attacks (timing, power analysis) remain possible. Mitigations exist but add complexity.
- Ecosystem maturity: Confidential computing for AI is still emerging. Production deployments exist but tooling and best practices are evolving rapidly.
- Attestation complexity: Setting up and verifying attestation requires careful implementation. Incorrect attestation provides false security.
Root Cause Analysis: Principles and Benefits
Master root cause analysis techniques for systematic problem solving. In collaboration with Starweaver.
Start on Coursera →Getting Started
- Identify candidates: Which AI workloads process data that's sensitive enough to warrant confidential computing? Start with the highest-sensitivity use cases.
- Choose your platform: Azure Confidential Computing, GCP Confidential VMs, or AWS Nitro Enclaves — each offers different TEE technologies
- Proof of concept: Deploy a non-critical AI workload in a confidential VM. Measure performance overhead and operational complexity.
- Attestation setup: Implement and verify attestation flow — this is the critical step that provides the actual security guarantee
- Production deployment: Migrate highest-sensitivity workloads with monitoring for performance impact
Confidential computing doesn't replace good security practices — it adds a hardware-backed layer of protection for data during processing. For AI workloads on sensitive data, it's increasingly the expected standard, not a nice-to-have.