Skip to main content
← All posts·
AI Infrastructure

Design for Topology Evolution: Infrastructure That Adapts Over 2 Years, Not 2 Months

Most infrastructure is designed for today's requirements and breaks when topology changes — new regions, hybrid cloud, edge compute, GPU clusters, sovereignty zones. Here's how to architect systems that evolve gracefully as your topology shifts over the next 1-2 years.

Luca Berton15 min read

Your infrastructure topology will change. Not "might change" — will. In the next 12-24 months, at least one of these will happen to your organisation:

  • A new regulation (DORA, NIS2, AI Act) requires data to stay in a specific jurisdiction → new sovereignty zone
  • An AI initiative demands GPU capacity you can't get from your current cloud provider → new cloud or on-prem GPU cluster
  • A customer or market expansion requires low-latency presence in a new region → new region deployment
  • A cost optimisation review reveals you're overpaying for cloud compute → hybrid cloud or repatriation
  • An edge AI use case (manufacturing, retail, autonomous systems) requires compute at the edge → new edge topology

Most infrastructure is designed for today's topology and breaks when any of these changes happen. The networking assumes single-region. The CI/CD pipeline deploys to one cluster. The monitoring is configured for one environment. Scaling to a new topology means re-architecting everything — 6-month project, 3 engineers, and it's still fragile.

The alternative: design for topology evolution from day one. Build the abstractions that make adding a new region, a new cloud, or a new edge location a configuration change — not an architecture change.

The Topology Evolution Map

Plot your likely topology changes on a 2-year timeline:

Topology Evolution Map (2026-2028)

Current (Q1 2026):

1 cloud provider (AWS eu-west-1)

1 Kubernetes cluster

Single-region, single-cloud

Year 1

— Likely Changes:

Q2

Add GPU node pool (A100/H100) for AI workloads. Same cluster, new node pool, GPU scheduling

Q3

EU data sovereignty zone (separate namespace or cluster). DORA/NIS2 requires isolated compute for financial data

Q4

Second region (eu-central-1) for DR/HA. Multi-region networking, data replication, failover

Year 2

— Probable Changes:

Q1

Second cloud provider (Azure/GCP) for specific services. Multi-cloud networking, unified deployment, cost arbitrage

Q2

On-prem GPU cluster for sensitive AI workloads. Hybrid connectivity, unified monitoring, workload placement

Q3

Edge deployment for retail/manufacturing AI inference. Edge cluster management, model distribution, offline capability

Q4

Neocloud provider for sovereign AI compute. EU sovereign cloud integration for regulated AI workloads

Design question — Which of these changes would require

re-architecting your current infrastructure?

Answer — All of them, if you designed for today only.

Answer — None of them, if you designed for topology evolution.

The Abstraction Layers

Topology-resilient infrastructure is built on abstraction layers that decouple application logic from infrastructure topology:

Layer 1: Cluster Abstraction

Applications shouldn't know which cluster they're running on. The platform should place workloads based on requirements, not hardcoded cluster names:

  • Cluster fleet management: Tools like Rancher, OpenShift ACM, or Cluster API manage multiple clusters as a fleet. Adding a new cluster is registering it with the fleet manager, not reconfiguring every application
  • Workload placement policies: "This workload needs GPU nodes" or "this workload must run in eu-west-1" or "this workload requires sovereign compute" — expressed as labels and constraints, not cluster-specific deployment configs
  • Federated deployment: ArgoCD ApplicationSets can deploy to multiple clusters from a single Git source. Adding a new cluster means adding it to the cluster list — existing applications deploy automatically if they match the placement criteria
# ArgoCD ApplicationSet: deploy to all clusters matching criteria
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: inference-api
spec:
  generators:
    - clusters:
        selector:
          matchLabels:
            has-gpu: "true"
            region: "eu"
            compliance: "dora"
  template:
    metadata:
      name: 'inference-api-{{name}}'
    spec:
      project: ml-team
      source:
        repoURL: https://git.example.com/ml-team/inference-api
        targetRevision: main
        path: deploy/
      destination:
        server: '{{server}}'
        namespace: ml-team-production

When you add a new GPU cluster in the EU with DORA compliance labels, the inference API deploys automatically. Zero application changes.

Layer 2: Network Abstraction

Multi-topology networking is where most infrastructure breaks. Design for it early:

  • Service mesh (Istio/Linkerd): Service-to-service communication abstracted from network topology. Services call each other by name, not IP. The mesh handles cross-cluster routing, mTLS, and load balancing
  • Multi-cluster service discovery: Services in cluster A can discover and call services in cluster B without hardcoded endpoints. Istio multi-cluster, Cilium ClusterMesh, or Submariner provide this
  • Ingress abstraction: External traffic routing through a cloud-agnostic ingress layer. Gateway API (Kubernetes standard) provides a portable ingress definition that works across clouds
  • DNS-based failover: Global DNS (Route53, Cloudflare) with health checks. If a region fails, traffic automatically routes to the surviving region. Application doesn't need to know

Layer 3: Data Abstraction

Data is the hardest layer to make topology-resilient because it has physics — latency, consistency, and sovereignty constraints:

  • Data classification: Every dataset tagged with sovereignty requirements (EU-only, specific country, unrestricted), latency requirements (local, regional, global), and consistency requirements (strong, eventual)
  • Storage abstraction: Crossplane or similar to provision storage per-topology. PostgreSQL in eu-west-1 for primary, read replica in eu-central-1 for DR, object storage in sovereign cloud for regulated data. Application connects via service name, not endpoint
  • Replication policies: Declarative data replication. "This database replicates to all EU clusters within 5 minutes." "This model registry syncs to edge clusters daily." "This audit log replicates to the sovereign zone in real-time"
  • Sovereignty enforcement: Data residency policies enforced at the infrastructure layer. Data tagged "eu-only" cannot be replicated outside EU topology — enforced by the platform, not by application developers remembering the rule

Layer 4: Observability Abstraction

  • Unified observability: One Grafana instance, one alerting pipeline, regardless of how many clusters, clouds, or regions. Thanos or Mimir for multi-cluster Prometheus federation. Loki for multi-cluster log aggregation
  • Topology-aware dashboards: Dashboards that show per-cluster, per-region, and aggregate views. When a new cluster joins the fleet, it appears in dashboards automatically
  • Cross-topology tracing: Distributed tracing that follows requests across clusters and regions. When a request goes from eu-west-1 to the GPU cluster and back, the trace shows the full journey

Layer 5: Security Abstraction

  • Federated identity: One identity provider, one RBAC model across all topologies. Engineer's permissions apply consistently whether they're accessing the primary cluster, GPU cluster, or edge cluster
  • Policy federation: Kyverno/OPA policies deployed to all clusters from a central policy repository. New cluster inherits the full policy set automatically. Security deploys with the topology
  • Certificate management: Shared CA with per-topology intermediate CAs. cert-manager federation for consistent TLS across all clusters
  • Secrets management: Vault with per-topology auth backends. Secrets accessible based on workload identity and topology constraints (some secrets are topology-specific)
📘 Book

Kubernetes Recipes

Practical guide for container orchestration and deployment — hands-on patterns you can use today.

View on Amazon →

The Evolution Playbook

You don't build all 5 abstraction layers on day one. You build the foundations and extend as topology changes happen:

Topology Evolution Playbook

Phase 1

— Single Cluster (Now):

Build:

✓ GitOps (ArgoCD) — deploys from Git, not kubectl

✓ Service mesh (Istio/Linkerd) — even single-cluster

✓ Policy-as-code (Kyverno) — from a policy repo

✓ External Secrets (Vault) — centralised secret management

✓ Data classification — tag datasets now, enforce later

Why now — These are all best practices for single-cluster too.

Zero overhead for future topology changes.

Phase 2

— Multi-Node Pool (GPU, +3 months):

Extend:

+ GPU node pool with scheduling labels

+ Node affinity/anti-affinity for workload placement

+ GPU monitoring (DCGM exporter)

Effort — 1-2 weeks. Abstractions from Phase 1 make this trivial.

Phase 3

— Multi-Region (+6-9 months):

Extend:

+ Second cluster in new region

+ ArgoCD ApplicationSets for fleet deployment

+ Multi-cluster service mesh (Istio multi-cluster)

+ Database replication (primary + read replica)

+ DNS-based failover

+ Multi-cluster Prometheus (Thanos/Mimir)

Effort — 4-6 weeks. Would be 4-6 months without Phase 1.

Phase 4

— Hybrid/Multi-Cloud (+12-18 months):

Extend:

+ On-prem or second cloud cluster joins fleet

+ VPN/interconnect networking

+ Crossplane for multi-cloud provisioning

+ Workload placement policies (cost, latency, sovereignty)

Effort — 6-8 weeks. Fleet management from Phase 3 scales.

Phase 5

— Edge (+18-24 months):

Extend:

+ Edge cluster management (lightweight K3s/MicroShift)

+ Model distribution pipeline (hub → edge sync)

+ Offline capability and store-and-forward

+ Edge-specific monitoring (bandwidth-aware)

Effort — 8-10 weeks. More complex, but foundations support it.

Decisions You Make Now That Matter Later

These specific decisions today determine whether topology evolution is easy or painful:

  • Use GitOps now: If deployments go through Git and ArgoCD today, adding a new cluster is adding a target. If deployments are manual kubectl, adding a cluster doubles your manual work
  • Use service mesh now: Even on a single cluster, service mesh gives you mTLS, observability, and service discovery abstractions. When you go multi-cluster, the mesh extends — you don't retrofit it
  • Classify data now: Tag every dataset and database with sovereignty and residency metadata today, even if you only have one region. When regulations require data isolation, you know exactly what needs to move
  • Abstract secrets now: If applications read secrets from Vault via External Secrets Operator today, moving to a new topology means configuring Vault auth for the new topology — not changing application configs
  • Use Kubernetes-native APIs: Gateway API instead of cloud-specific load balancers. Crossplane instead of cloud-specific Terraform. cert-manager instead of cloud-specific certificate services. Portability is a design choice, not a migration project
🎓 Course

Federated Learning and Privacy-preserving RAGs

Implement secure AI models using federated learning techniques.

Start on Pluralsight →

What Open Empower Delivers

  • Topology evolution assessment: Map your likely topology changes over 24 months. Identify which current decisions will block evolution and which abstractions to prioritise
  • Foundation architecture: GitOps, service mesh, policy federation, secrets management, and data classification designed for single-cluster today and multi-topology tomorrow
  • Multi-cluster expansion: ArgoCD ApplicationSets, multi-cluster networking, database replication, and federated observability when you're ready to grow
  • Hybrid and multi-cloud: Crossplane provisioning, workload placement policies, and cost-optimised topology selection across clouds and on-prem
  • Edge deployment: Lightweight Kubernetes (K3s/MicroShift), model distribution, offline capability, and edge-specific monitoring for edge AI use cases

Design for the topology you'll have in 2 years, not the one you have today. Let's map your evolution.

infrastructure topology
architecture evolution
multi-cloud
hybrid cloud
edge computing
gpu clusters
sovereignty
platform engineering
kubernetes
future-proofing

Need help applying this in your organization?

Get a free 30-minute assessment with actionable recommendations — whether we work together or not.

Book Your Free AI Platform Assessment

Or see AI readiness assessment scope & pricing

18+ years experience · Ex-Red Hat & Dell · Speaker at KubeCon EU 2026

Luca Berton

Written by

Luca Berton

CEO at Open Empower. 18+ years building enterprise infrastructure at JPMorgan Chase, Red Hat & Dell. Author of 9 technical books. Speaker at Red Hat Summit and KubeCon EU 2026. Instructor on Coursera, Pluralsight & Udemy.

Get more insights like this

Practical AI infrastructure and platform engineering guides — delivered to your inbox.

Subscribe to Newsletter →