Trace, Span, Attribute, Status

A span is a small, fixed record. Trace id groups it with everything else caused by the same request. Span id identifies it. Parent span id points at the work that caused it, which is what makes the tree. Start time and duration place it on the timeline. Status says whether the operation succeeded. Attributes are typed key/value facts about *this* operation. Everything expensive about tracing follows from the fact that this record is emitted once per operation per request.

Two fields carry more weight than people expect. Name should be a low-cardinality operation identifier — GET /orders/{id}, not GET /orders/8812 — because the name is what you group by; putting the id in the name gives you a million distinct operations and no aggregation, the same cardinality mistake covered in Cardinality: The Label That Took Down Monitoring. Status should be set deliberately: a span that returned HTTP 404 is usually not an error of the span, while a span that threw is. Sloppy status handling makes error-rate queries over traces useless.

Span events are the underused field. An event is a timestamped annotation *inside* a span — "connection acquired", "retry 1 scheduled", "cache miss" — which lets you explain a 300 ms span without creating three more spans. Events are cheaper than child spans and better than logs for anything whose only meaning is relative to this operation.

Kind tells the backend what role this span played: SERVER (this process handled an inbound request), CLIENT (it called out and waited), PRODUCER / CONSUMER (it put work on or took work off a queue), INTERNAL (local work worth timing). The pairing matters: a CLIENT span in one service and the SERVER span it caused in another are the two halves of one network hop, and the difference between their durations is exactly the network plus queueing time — which is often where a mystery 60 ms lives.

Producer/consumer is the pair that trips teams up, because a queued job is not a child of the request that enqueued it in any useful sense: the request finished, possibly minutes earlier. Modelling it as a child produces a trace whose root span is somehow 4 minutes long. The correct shape uses a link instead, which is the subject of Parents, Children and Links.

Getting kinds right is not bureaucracy — backends use them to compute service maps, to decide what counts as an entry point for RED metrics, and to detect the caller/callee latency gap above. Kind-less spans still render, and quietly disable half your tooling.

The reflex to put everything on the span is expensive and, for some fields, dangerous. Attributes are stored per span per request: a 2 KB serialized request body on a span emitted 8,000 times a second is 16 MB/s of telemetry to teach you nothing you could not have sampled. Worse, request and response bodies are exactly where bearer tokens, session cookies and personal data live, and a tracing backend is usually readable by a far wider audience than your production database — the same trap as What You Just Wrote Into a Log Half the Company Can Read.

The useful rule is to ask what the field is *for*. If you will group or filter by it and it has a bounded set of values (route, table, cache hit/miss, tenant tier, model name), it is an attribute. If it is a timestamped thing that happened during the operation, it is a span event. If it is a full payload or a long free-text message, it belongs in a log correlated by trace id — logs are searched, not aggregated, and that is the right access pattern for bulk detail. If you need it aggregated across all requests including unsampled ones, it must be a metric, because sampled traces cannot produce trustworthy totals.

That last point is the one that bites teams late. Traces are sampled; metrics are not. Any number you plan to alert on, put in an SLO, or bill against has to come from a metric, no matter how faithfully your spans record it (Sampling Without Throwing Away the Evidence).