Jaeger: Traces, Storage and Deployment

Jaeger

Jaeger collects, stores, and helps you investigate distributed traces. Use it with instrumentation and propagation that preserve relationships across services, then verify that the expected trace survives collection, storage, and querying.

An attractive waterfall cannot establish that every request was sampled or every dependency was instrumented. Start with a known request and an expected span relationship before drawing architectural conclusions from the traces you can find.

TL;DR

OpenTelemetry instrumentation creates spans and propagates context; Jaeger can receive traces through OTLP.
Jaeger v2 uses one configurable binary with roles such as collector, query, and ingester; follow version-matched configuration.
A development in-memory backend is not durable production storage.
A service graph summarizes observed evidence and has view-specific meaning; an absent edge is not proof of no dependency.
Troubleshoot a known trace across sampling, export, queues, storage, and query filters rather than treating an empty search as a single failure mode.

Quick Example

In a disposable Python environment, install opentelemetry-sdk and record its version. Save this as trace_example.py and run python trace_example.py. It creates two fictional service resources in one process, propagates a trace context through a dictionary, and captures spans in memory. It does not send data to Jaeger or make a network request.

The expected message confirms the declared relationship in SDK output. The always-on sampler is a fixture choice, not a production sampling recommendation. OpenTelemetry's propagation guide describes injecting context at the sender and extracting it at the receiver. A real HTTP or queue integration must carry that context through the actual transport and trust boundary.

Core Concepts

Instrumentation, propagation, and export

Instrumentation records operations as spans. Propagation links work across boundaries. Export sends completed telemetry toward the backend. Missing any one of these can produce incomplete traces even when the others are functioning.

Jaeger's feature documentation identifies OTLP ingestion support. That does not mean enabling Jaeger automatically instruments an application. Select supported SDKs or instrumentation, configure the intended export path, and verify service identity and trace context on a known request.

Collector, storage, and query

The v2 architecture distinguishes collection, query, and optional ingestion roles. A collector receives trace data and writes toward storage; query serves the APIs and UI used to investigate it. A Kafka-buffered arrangement adds an ingester rather than changing the need for sustainable storage capacity.

Think in stages rather than one box labeled tracing. A successful application export can still be followed by a queue overflow or storage write failure. A healthy query process cannot display data that never reached the selected backend.

Sampling and retention shape what you see

Record the sampling decisions and retention window that apply to an investigation. An omitted trace, an expired trace, and a trace filtered out by the search form are different situations.

When assessing a latency problem, remember that the visible traces may not represent every request equally. State what the evidence contains and correlate with appropriate request metrics and logs before treating a small trace sample as a complete traffic distribution.

Read Service Graphs with Their Scope

Jaeger's topology documentation distinguishes the classic System Architecture view from the Deep Dependency Graph. In the classic view, observed A-to-B and B-to-C edges do not establish that one trace contains the complete A-to-B-to-C path. The deeper view has its own focal-service and trace-selection scope.

Use a graph as a route into evidence. Open representative traces, inspect operations and service identities, and verify the instrumentation boundary. Missing instrumentation, sampling, asynchronous work, or a selected time window can hide an edge. A graph is not an authoritative inventory of every deployed connection.

For asynchronous processing, decide which relationship the operation should express and test how context survives the queue. A new consumer span with an unrelated trace identifier may be intentional in some designs, but it should not be an accidental result of dropping propagation metadata.

Deploy the Pipeline You Can Operate

Jaeger v2's deployment guide uses explicit YAML configuration and allows the same binary to serve different roles. Use documentation matching the selected release; copying a legacy all-in-one command can omit configuration or imply outdated component assumptions.

Choose a supported storage backend from durability, query requirements, expected traffic, and operational support. An in-memory development setup loses data on restart. A persistent backend still needs capacity planning, retention management, access controls, and an upgrade/recovery procedure.

Keep trace intake and query access on approved network and authentication boundaries. Attributes can contain sensitive values, so define a collection policy before forwarding arbitrary headers or request bodies. A trace identifier is useful for correlation; it is not an access-control mechanism.

Best Practices

Keep a known-trace acceptance test

Generate a controlled request with expected service names and parent relationships. Verify it through the real pipeline after deployment or configuration changes. Local SDK success, as in the example, verifies only the first part.

Monitor the tracing system itself

Inspect export, queue, receiver, storage, and query health. A telemetry backend can degrade while the application still serves traffic, leaving investigators with misleading gaps.

Treat service identity as a stable convention

Use consistent resource naming so one service does not fragment into unrelated names across releases. Keep instance identity and high-cardinality request data separate from the service grouping used for investigation.

Common Mistakes

Bad: Assume an empty search proves the application received no requests.

Correct: Check sampling, time filters, export failures, processing, storage, and retention using a known request.

Bad: Treat a service graph as proof of every transitive dependency.

Correct: Read the selected view's semantics and inspect traces that support the relationship.

Bad: Deploy an in-memory example and expect evidence to survive restarts.

Correct: Choose persistent storage and operate its retention and recovery requirements deliberately.

FAQ

Does Jaeger replace OpenTelemetry?

They can serve complementary roles: OpenTelemetry creates and transports telemetry, while Jaeger provides trace ingestion, storage integration, and investigation. Define the actual pipeline rather than treating the names as competing switches.

Does this example verify Jaeger ingestion?

No. It verifies SDK propagation and parent relationships in memory. Run a known trace through the configured OTLP endpoint and storage/query path before claiming backend integration.

Do I need Kafka?

Not for every deployment. Evaluate buffering and decoupling needs against complexity and storage capacity. A buffer does not remove a sustained throughput mismatch.

Can traces contain confidential data?

Yes. Define allowed attributes, access, and retention, and verify filtering at the relevant collection boundary before sending real application data.