The Paved Path to Observability

The Paved Path to Observability Kasper Borg Nissen, Principal Developer Advocate at @phennex kaspernissen.xyz

Who? Kasper Borg Nissen Principal Developer Advocate at Dash0 KubeCon+CloudNativeCon EU/NA 24/25 Co-Chair (former) CNCF Ambassador Golden Kubestronaut CNCG Aarhus, KCD Denmark Organizer Co-founder & Community Lead Cloud Native Nordics

https://university.platformengineering.org/observability-for-platform-engineering

Observability should feel invisible, but never be absent @phennex kaspernissen.xyz

@phennex kaspernissen.xyz

@phennex kaspernissen.xyz

Why Platform Engineering? @phennex kaspernissen.xyz

A digital platform is a foundation of self-service APIs, tools, services, knowledge and support arranged as a compelling internal product. - Evan Bottcher @phennex kaspernissen.xyz

Platform Engineering Principles Self-Service Experience Explicit and Consistent APIs Golden Paths Modularity Platform as a Product Core Requirements @phennex kaspernissen.xyz

How does this apply to Observability? @phennex kaspernissen.xyz

@phennex kaspernissen.xyz

This fragmentation, leads to @phennex kaspernissen.xyz Image by pngtree.com

This fragmentation, leads to Complex Query Languages @phennex kaspernissen.xyz Image by pngtree.com

This fragmentation, leads to Complex Query Languages @phennex kaspernissen.xyz Vendor lock-in Image by pngtree.com

This fragmentation, leads to Complex Query Languages @phennex kaspernissen.xyz Vendor lock-in Metadata Inconsistency Image by pngtree.com

This fragmentation, leads to Complex Query Languages Vendor lock-in Metadata Inconsistency No instrumentation due to high complexity @phennex kaspernissen.xyz Image by pngtree.com

This fragmentation, leads to Complex Query Languages Vendor lock-in No instrumentation due to high complexity @phennex kaspernissen.xyz Metadata Inconsistency Lack of unified insights Image by pngtree.com

As a Platform Engineer, you have a dual role 01 02 Observing the platform Enable developers (cloud, cluster, CI/CD, shared DBs, etc) @phennex kaspernissen.xyz (auto-instrumentation, logs, metrics, traces)

01 Observe the platform @phennex kaspernissen.xyz Image by pngtree.com

The foundation @phennex kaspernissen.xyz

OpenTelemetry OpenTelemetry OTel is an open source project designed to provide standardized tools and APIs for generating, collecting, and exporting telemetry data such as traces, metrics, and logs The de-facto standard for distributed tracing, also supports metrics and logs (soon profiling and RUM The main goals of the project are: ● ● ● Unified telemetry Vendor-neutrality Cross-platform @phennex kaspernissen.xyz

1/1/20241/1/2025 Commits: 27.168 PRs+Issues: 58.508 Source: CNCF Velocity Commits: 44.486 PRs+Issues: 56.299

Signals METRICS 42 LOGS 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 @phennex kaspernissen.xyz TRACES PROFILES RUM

Not pillars … Metrics @phennex kaspernissen.xyz Logs Traces

Not pillars … We donʼt have a metrics problem, or a tracing problem. We have systems problems. Metrics @phennex kaspernissen.xyz Logs Traces

Correlation is the superpower METRICS 42 LOGS 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 20/JUN/2025 “GET / HTTP/1.1ˮ 200 @phennex kaspernissen.xyz TRACES PROFILES RUM

02 Enable developers @phennex kaspernissen.xyz Image by pngtree.com

No-touch instrumentation on Kubernetes Instrumentation Instructs how to inject auto-instrumentation Injects instrumentation in to the pod OpenTelemetry Operator @phennex kaspernissen.xyz

instrumentation.opentelemetry.io/inject-go: “<ns>/<instrumentation>” instrumentation.opentelemetry.io/inject-java: “<ns>/<instrumentation>” instrumentation.opentelemetry.io/inject-dotnet: “<ns>/<instrumentation>” instrumentation.opentelemetry.io/inject-nodejs: “<ns>/<instrumentation>” instrumentation.opentelemetry.io/inject-python: “<ns>/<instrumentation>” @phennex kaspernissen.xyz

OpenTelemetry: the standard plumbing Instrumentation OTel API & SDK Telemetry Backends The OpenTelemetry Collector auto-instrumentation Time-series database … Log database Receive Process Analysis Tools Export Trace database Infrastructure … Kubernetes … Generate and Emit @phennex transmit kaspernissen.xyz Collect, Convert, Process, Route, Export transmit Store & Analyze

OpenTelemetry: the standard plumbing OTel API & SDK auto-instrumentation Vendor space Collection of Telemetry is The OpenTelemetry Collector standardized … Receive Process Export Infrastructure Kubernetes … “The last observability agent you will ever installˮ … and many more. Generate and Emit @phennex transmit kaspernissen.xyz Collect, Convert, Process, Route, Export transmit Store & Analyze

Observability as code An open specification for dashboards. CNCF Sandbox project @phennex kaspernissen.xyz

Putting it all together… @phennex kaspernissen.xyz

@phennex kaspernissen.xyz

@phennex kaspernissen.xyz

@phennex kaspernissen.xyz

@phennex kaspernissen.xyz

@phennex kaspernissen.xyz

@phennex kaspernissen.xyz

Putting it all together… @phennex kaspernissen.xyz

Golden defaults for developers @phennex kaspernissen.xyz

Observability is evolving - fast. @phennex kaspernissen.xyz

OpenTelemetry is standardizing telemetry collection. @phennex kaspernissen.xyz

Applying Platform Engineering principles can transform observability into a seamless, scalable, and developer-friendly experience. @phennex kaspernissen.xyz

Treat Observability as a platform product @phennex kaspernissen.xyz

Thank you! Get in touch! Kasper Borg Nissen, Principal Developer Advocate at Demo can be found here! https://github.com/dash0hq/otel-platform-demo @phennex kaspernissen.xyz