183-observability-tracing-opentelemetry - Claude MCP Skill

# Java Distributed Tracing with OpenTelemetry

Implement robust distributed tracing in Java with OpenTelemetry by modeling meaningful spans, preserving context propagation, and instrumenting critical business and infrastructure paths with low-overhead, privacy-safe telemetry.

**What is covered in this Skill?**

- OpenTelemetry tracing fundamentals for Java services
- Span design: boundaries, parent/child relationships, and operation naming
- Context propagation across HTTP, messaging, async tasks, and thread boundaries
- Semantic conventions and stable attribute naming
- Error/status/event recording best practices
- Sampling strategy and performance/cost trade-offs
- Privacy and security controls for trace attributes
- Testing and verification of trace propagation and span correctness

**Scope:** Distributed tracing quality in application and integration layers, focused on diagnosability, consistency, and operational safety.

## Constraints

Tracing instrumentation must preserve context correctly and avoid leaking sensitive data. Over-instrumentation and high-cardinality attributes can harm cost and signal quality.

- **PROPAGATION FIRST**: Ensure context propagation across all sync/async boundaries before adding extra span detail
- **NO SENSITIVE DATA**: Never store secrets, credentials, tokens, raw payloads, or PII in span attributes/events
- **LOW CARDINALITY ATTRIBUTES**: Avoid unbounded values in attributes that are used for aggregation/search
- **VERIFY**: Run `./mvnw clean verify` or `mvn clean verify` after applying tracing changes

## When to use this skill

- Improve tracing
- Apply OpenTelemetry tracing
- Add distributed tracing
- Refactor tracing instrumentation

## Workflow

1. **Define trace model and critical flows**

Identify high-value request and async flows, define operation boundaries, and choose span names/attributes aligned with semantic conventions.

2. **Instrument and propagate context**

Add OpenTelemetry spans to key boundaries and ensure trace context is propagated across HTTP clients/servers, messaging, and executor-based async work.

3. **Harden span data and sampling**

Record status/errors/events consistently, remove sensitive data, control attribute cardinality, and configure sampling/exporters according to environment needs.

4. **Validate traces end-to-end**

Verify parent-child relationships, propagation continuity, and backend visibility through tests and runtime checks.

## Reference

For detailed guidance, examples, and constraints, see [references/183-observability-tracing-opentelemetry.md](references/183-observability-tracing-opentelemetry.md).