Luca BertonDo You Actually Need Kubernetes?
This might seem like a strange question from an AI infrastructure consultancy that runs on Kubernetes. But we've seen too many enterprises adopt Kubernetes when Nomad — or even a managed container service — would have been a better fit. The right orchestrator depends on your workloads, team, and operational maturity.
Kubernetes vs Nomad: Quick Answer
Quick answer: Kubernetes wins for AI/ML workloads, microservices at scale, and teams that need the CNCF ecosystem. Nomad wins for mixed workloads (containers, VMs, batch jobs) and small ops teams that want operational simplicity. Whether you're comparing Kubernetes vs Nomad or Nomad vs Kubernetes, the answer depends on the same factors: workload mix, team size, and how much operational overhead you can absorb — covered in detail below.
Honest Comparison
| Dimension | Kubernetes | Nomad |
|---|---|---|
| Complexity | High (steep learning curve) | Moderate (single binary, HCL config) |
| Workload types | Containers (primary), VMs via KubeVirt | Containers, VMs, Java, raw exec, system tasks |
| Ecosystem | Massive (CNCF, thousands of tools) | Smaller (HashiCorp ecosystem + community) |
| Managed options | EKS, AKS, GKE (all major clouds) | HCP Nomad (HashiCorp Cloud) |
| Networking | CNI plugins, service mesh (Istio/Cilium) | Consul Connect (service mesh) |
| Storage | CSI drivers (comprehensive) | CSI support + host volumes |
| GPU support | Device plugins, NVIDIA GPU Operator | Device plugins (simpler model) |
| Scaling | HPA, VPA, KEDA, cluster autoscaler | Built-in autoscaling, simpler model |
| Licence | Apache 2.0 | BSL 1.1 (since 2023) |
Kubernetes Recipes
Practical guide for container orchestration and deployment — hands-on patterns you can use today.
View on Amazon →When Kubernetes Is the Right Choice
- Microservices at scale — 50+ services with complex networking, service mesh, and traffic management needs
- AI/ML workloads — GPU scheduling, distributed training, model serving with auto-scaling — Kubernetes' ecosystem is far richer
- Cloud-native ecosystem — If you want to use CNCF tools (Istio, Cilium, ArgoCD, Prometheus), Kubernetes is the assumed platform
- Talent availability — Kubernetes skills are far more common in the job market
- Multi-cloud — Kubernetes runs identically on every major cloud and on-premises
When Nomad Makes More Sense
- Mixed workloads — If you run containers alongside VMs, Java applications, and batch jobs, Nomad orchestrates all of them natively
- Operational simplicity — Small teams (< 5 infrastructure engineers) that can't afford the Kubernetes operational overhead
- HashiCorp ecosystem — If you're already invested in Consul, Vault, and Terraform, Nomad integrates seamlessly
- Edge deployments — Nomad's single binary and lower resource requirements work better in constrained environments
- Gradual modernisation — Teams migrating from VM-based to container-based architectures can run both in Nomad during transition
Automating IT Infrastructure with Ansible
Learn Ansible to automate IT operations and enhance system reliability.
Start on Udemy →The AI Infrastructure Angle
For AI workloads specifically, Kubernetes has a significant advantage:
- GPU scheduling: NVIDIA GPU Operator, MIG support, time-slicing — Kubernetes' GPU ecosystem is mature
- ML frameworks: KubeFlow, Ray on K8s, Seldon Core, KServe — the ML ops ecosystem assumes Kubernetes
- Distributed training: PyTorch Distributed, Horovod — Kubernetes operators manage distributed training jobs
- Model serving: KServe, Triton Inference Server — production model serving assumes Kubernetes
Nomad can run AI workloads, but you'll be building custom tooling where Kubernetes has off-the-shelf solutions.
Decision Framework
Choose Kubernetes if: AI/ML workloads, microservices at scale, need the CNCF ecosystem, have 5+ infrastructure engineers.
Choose Nomad if: Mixed workload types, small ops team, already in HashiCorp ecosystem, gradual modernisation path.
Consider managed Kubernetes (EKS/AKS/GKE) if: You want Kubernetes benefits without self-managing the control plane.
AI Readiness Checklist
50-point interactive checklist covering strategy, data, infrastructure, governance, and people. Score your organisation's AI readiness.
Get Free Checklist →