Luca BertonThe Vendor Lock-In Problem in Observability
Most observability solutions require proprietary agents and SDKs. Switch from Datadog to Grafana? Re-instrument every service. Move from New Relic to Dynatrace? Rewrite all your dashboards and alerts. For regulated enterprises that may need to change vendors due to data sovereignty requirements, contract changes, or cost optimisation, this lock-in is a strategic risk.
OpenTelemetry (OTel) solves this: one standard instrumentation SDK, one collector, any backend. Instrument once, export to any observability platform.
OpenTelemetry Components
The Three Signals
- Traces: Follow a request across services — identify latency sources and failure points
- Metrics: Counters, histograms, gauges — quantitative measurements of system behaviour
- Logs: Structured log events correlated with traces and metrics (newer, still maturing)
OTel's vision: all three signals, correlated, from a single instrumentation library. Click a trace span, see the related logs and metrics. This correlation is invaluable for incident investigation.
The OTel Collector
The Collector is the pipeline between instrumentation and backends:
- Receivers: Accept data via OTLP, Jaeger, Zipkin, Prometheus, and other protocols
- Processors: Filter, batch, sample, enrich, and transform telemetry data
- Exporters: Send data to any backend — Grafana (Mimir/Loki/Tempo), Datadog, New Relic, Elasticsearch, etc.
Compliance value: The Collector can filter sensitive data before it leaves your infrastructure. PII scrubbing, data classification, and routing decisions happen at the Collector level.
Auto-Instrumentation
OTel supports automatic instrumentation for many languages — add observability without code changes:
- Java: Java agent (javaagent JAR) — instruments Spring Boot, Quarkus, Micronaut, JDBC, HTTP clients automatically
- Python: Auto-instrumentation for Django, Flask, FastAPI, SQLAlchemy, requests
- Node.js: Auto-instrumentation for Express, Fastify, HTTP, gRPC
- .NET: Auto-instrumentation for ASP.NET Core, Entity Framework, HttpClient
- Go: Manual instrumentation required (no runtime agent), but comprehensive SDK
Kubernetes pattern: The OpenTelemetry Operator can inject auto-instrumentation into pods via annotations — zero application changes, zero Dockerfile changes.
Kubernetes Recipes
Practical guide for container orchestration and deployment — hands-on patterns you can use today.
View on Amazon →OTel for Compliance
- DORA Art. 10 (Monitoring): OTel provides the instrumentation layer for continuous monitoring of all ICT systems
- DORA Art. 17 (Incident Detection): Correlated traces + metrics + logs enable rapid incident detection and root cause analysis
- NIS2 Art. 21(2)(b): Incident detection capabilities built on reliable telemetry
- Data sovereignty: OTel Collector runs in your infrastructure. Telemetry data never leaves your control unless you explicitly export it.
- Vendor flexibility: Change observability backends without re-instrumenting any service — reduces third-party concentration risk (DORA Art. 28-44)
Getting Started
- Deploy the OTel Collector as a DaemonSet (per-node) or Deployment (centralised) in Kubernetes
- Install the OTel Operator for automatic instrumentation injection
- Auto-instrument one service — add the annotation, verify traces appear in your backend
- Configure PII filtering in the Collector pipeline — scrub sensitive attributes before export
- Roll out incrementally — auto-instrument service by service, starting with the most critical
Back-End Infrastructure: Servers, Secure APIs and Data
Build secure back-end infrastructure from the ground up. In collaboration with Starweaver.
Start on Coursera →