ML Observability in Production: Monitoring Beyond Accuracy

You shipped the model. Accuracy was 94% in testing. Everyone celebrated. Three months later, the model is silently making terrible predictions and nobody noticed because nobody was watching the right metrics.

ML observability in production is fundamentally different from model evaluation during development. Here's what you actually need to monitor — and why most teams get it wrong.

The Five Pillars of ML Observability

Pillar 1: Data Quality Monitoring

Data issues are the most common cause of production ML failures:

• Schema validation: Input data matches expected schema — types, ranges, cardinality. Detect upstream changes that break your pipeline.
• Completeness: Missing value rates by feature. A feature that was 99% complete during training but drops to 80% in production will degrade predictions.
• Distribution monitoring: Statistical tests (KS test, PSI) comparing production input distributions to training distributions
• Volume monitoring: Request volume anomalies — sudden drops (upstream system failure) or spikes (possible attack or error)
• Freshness: For features derived from data pipelines, monitor data freshness. Stale features produce stale predictions.

Pillar 2: Model Performance Monitoring

• Prediction distribution: Monitor the distribution of model outputs. Shifts in prediction distribution often indicate problems before ground truth is available.
• Confidence distribution: Changes in model confidence patterns signal data drift or model degradation
• Ground truth metrics: When ground truth becomes available (delayed labels), compute accuracy, precision, recall against production predictions
• Segment performance: Monitor performance across segments (geography, customer type, product category). Global metrics can hide segment-level degradation.
• Fairness metrics: Ongoing monitoring of prediction disparities across protected groups

Pillar 3: Infrastructure Monitoring

• Latency: p50, p95, p99 inference latency. Track separately: model inference time, preprocessing time, network time, total end-to-end.
• Throughput: Requests per second and tokens per second for LLMs
• Error rates: 4xx (client errors), 5xx (server errors), timeout rates
• Resource utilization: GPU memory, GPU compute, CPU, memory, disk for serving infrastructure
• Availability: Uptime of model endpoints against SLA targets

Pillar 4: Business Impact Monitoring

Model metrics are necessary but not sufficient. Connect ML to business outcomes:

• Business KPIs: Revenue, conversion rate, customer satisfaction — metrics the business actually cares about
• A/B test results: When running model experiments, track business metrics alongside model metrics
• Human override rate: How often do humans override model recommendations? Increasing override rates signal declining trust.
• User feedback: Explicit feedback (thumbs up/down) and implicit feedback (did the user follow the recommendation?)

Pillar 5: Compliance Monitoring

• Audit log completeness: All predictions logged with inputs, outputs, model version, and timestamp
• Bias monitoring: Automated fairness metric computation on rolling windows
• Data governance compliance: Input data used within defined consent boundaries and retention policies
• Model governance: Deployed model version matches approved version in model registry

Alerting Strategy

Tool Landscape

• Open source: Evidently (data/model monitoring), NannyML (performance estimation without ground truth), Prometheus + Grafana (infrastructure)
• Commercial: Arize, WhyLabs, Fiddler, DataDog ML Monitoring
• Cloud-native: SageMaker Model Monitor, Vertex AI Model Monitoring
• Custom: For specific business metric monitoring, custom dashboards integrating ML metrics with business data

The best ML observability setup is the one your team actually looks at. Start with 5-10 critical metrics on a single dashboard, set up actionable alerts for the top 3, and expand from there. A comprehensive monitoring system that nobody checks is worse than a simple one that's actively used.