Deterministic Scaffolding for Agentic AI: A Case Study in What Actually Works in Production

The fundamental tension in deploying agentic AI to production is simple: AI models are probabilistic; production systems must be deterministic. An AI agent might generate the correct API call 98% of the time. In a regulated enterprise processing 10,000 transactions per day, 2% means 200 incorrect operations — per day. That's not acceptable for a payment system, a medical device, or a compliance workflow.

The solution isn't to make AI models deterministic — that's fighting physics. The solution is deterministic scaffolding: infrastructure that wraps probabilistic AI capabilities in deterministic validation, orchestration, and rollback mechanisms. The AI proposes; the scaffolding disposes.

This article presents a real production architecture — anonymised but based on actual client engagements — showing how deterministic scaffolding makes agentic AI safe for regulated environments.

The Architecture: Agentic Document Processing for a Financial Institution

The Problem

A mid-sized European bank processes 2,000 regulatory filings per month. Each filing requires: extraction of key data points, classification by regulation type (DORA, CRA, MiFID II), cross-referencing against the bank's current compliance status, and generation of action items for the compliance team. Previously handled by 12 analysts working 3 weeks per month — expensive, slow, and error-prone (human error rate: ~5%).

The Naive Approach (What Doesn't Work)

The initial proposal: feed documents to GPT-4, have it extract data, classify regulations, and generate action items end-to-end. The problems:

• Hallucinated regulation references: The model sometimes cited regulations that don't exist or misattributed requirements to the wrong regulation
• Inconsistent extraction: The same document processed twice produced different data points. Regulators expect deterministic outputs for the same inputs
• No audit trail: "The AI said so" is not an acceptable answer when the financial regulator asks how you determined your compliance status
• Confidentiality: Regulatory filings contain non-public market data. Sending them to external AI APIs was a GDPR and DORA violation

The Deterministic Scaffolding Architecture

The production architecture separates AI capabilities from system-level guarantees:

Architecture: Deterministic Scaffolding for Regulatory Document Processing

Why This Works

The AI Never Touches the Output Directly

The AI generates proposals. The scaffolding validates them. The difference is critical:

• If the AI hallucinates a regulation reference, the validation layer catches it (deterministic lookup against the regulation database)
• If the AI extracts inconsistent data points, the triple-processing consistency check catches it
• If the AI's confidence is low, the human review layer catches it
• If the AI produces malformed output, the schema validation catches it

The AI's probabilistic nature is constrained by deterministic checks at every stage. The system fails safe — a validation failure routes to human review, it never produces unvalidated output.

Self-Hosted for Sovereignty

The entire AI stack runs on the bank's own OpenShift AI cluster. No document data leaves the bank's infrastructure. The model (Llama 3 70B) runs on dedicated GPU nodes with network isolation — the inference pods can reach the document store and the validation database, but cannot reach the internet. This satisfies DORA's ICT risk management requirements and GDPR's data residency requirements simultaneously.

Audit Trail by Design

Every step produces an immutable audit record:

• Document received: hash, timestamp, source, classification
• AI extraction: model version, prompt version, raw output, processing time
• Validation results: each check passed/failed, specific failures, confidence scores
• Human review: reviewer identity, decision (approved/corrected/rejected), corrections made
• Final output: action items written, cross-references verified, compliance register updated

When the regulator asks "how did you determine that this filing requires action X?", the answer is a traceable chain from document to output — not "the AI told us."

Results

• Processing time: 3 weeks (12 analysts) → 3 days (2 analysts reviewing AI output)
• Cost: ~70% reduction in analyst hours
• Error rate: 5% (human-only) → 0.3% (AI + scaffolding + human review). The scaffolding catches most AI errors; human reviewers catch the rest
• Audit trail: Complete, machine-readable, traceable — previously non-existent
• Regulatory satisfaction: The financial regulator reviewed the architecture and approved it — specifically because the AI never produces unvalidated output

Patterns That Generalise

This architecture isn't specific to regulatory document processing. The deterministic scaffolding pattern applies wherever you're deploying AI in regulated or high-stakes contexts:

• AI-assisted code review: AI proposes code changes → static analysis validates → test suite verifies → human approves merge
• AI-assisted customer support: AI drafts response → policy compliance check → tone analysis → human approves or edits → response sent
• AI-assisted fraud detection: AI flags suspicious transactions → rule engine validates flags → risk score calculated deterministically → human reviews high-score cases
• AI-assisted infrastructure management: AI proposes scaling changes → policy engine validates against resource limits and budget → change request created → human or auto-approval based on risk tier

The Pattern

1. Decompose the task into AI-suitable subtasks (extraction, classification, generation) and deterministic subtasks (validation, cross-referencing, output formatting)
2. Sandwich AI between deterministic layers: Input validation → AI processing → Output validation → Human review (for low-confidence) → Deterministic output
3. Triple-process for consistency: Run the same input through the AI multiple times. Flag divergent outputs. Consistent outputs have higher confidence
4. Structured outputs only: Never let AI produce free-text that feeds directly into downstream systems. JSON with schema validation. Every field typed, every reference verifiable
5. Fail safe: When validation fails, route to humans. Never produce unvalidated output. The system degrades to human processing, not to uncontrolled AI output

Infrastructure Requirements

Deterministic scaffolding requires specific infrastructure:

• Self-hosted inference: For regulated data, external APIs aren't an option. GPU scheduling on Kubernetes with namespace isolation — exactly the multi-tenant GPU patterns I'll discuss at KubeCon Europe 2026
• Validation database: An authoritative data store for reference checking (regulation database, product catalogue, compliance register). Must be deterministic and versioned
• Workflow orchestration: Argo Workflows, Temporal, or similar for orchestrating the multi-step pipeline with retry logic, timeout handling, and human-in-the-loop integration
• Audit log infrastructure: Immutable, append-only storage for the complete processing chain. OpenSearch or similar for queryable audit trails
• Human review interface: A UI where reviewers see the AI's output, the validation results, and the confidence scores — with the ability to approve, correct, or reject

For the Kubernetes deployment patterns that support this architecture — Jobs for batch processing, PVCs for document storage, NetworkPolicies for inference isolation — see Kubernetes Recipes.

What Open Empower Delivers

We design and implement deterministic scaffolding architectures for regulated enterprises:

• Architecture design: Decompose your AI use case into AI subtasks and deterministic subtasks, with validation at every boundary
• Self-hosted inference: GPU infrastructure on OpenShift AI/Kubernetes with namespace isolation and network segmentation
• Validation layer: Schema validation, reference checking, consistency verification, confidence scoring
• Workflow orchestration: Multi-step pipelines with human-in-the-loop integration and full audit trails
• Regulatory alignment: Architecture mapped to DORA, CRA, AI Act, and GDPR requirements

AI is powerful. Unscaffolded AI is unreliable. The scaffolding is what makes it production-ready. Let's architect yours.