Trace timeline

The trace timeline shows where a single request, a trace, spends its time as it moves through many spans. Spans are stacked into a parent–child tree, and each span's start offset and duration are drawn as a horizontal bar. Bottlenecks, ordering and nesting can then be read at a glance.

The fullscreen trace timeline overlay. A toolbar holds a Service filter, a duration range and a span search. Below it, a filter bar has Errors, Queries and Events checkboxes and three span-group checkboxes. The body is a span tree on the left with timing bars against a millisecond axis, and an attributes sidebar on the right for the selected span.

The implementation is the TraceTimeline component.

Preview and fullscreen

The timeline comes in two stages: a timeline preview that shows it in context, and a fullscreen overlay for investigation.

The timeline preview: a box titled Trace timeline with an All spans button, three spans with their timing bars, and a link reading "+5 more in the fullscreen overlay".

Timeline preview, TraceTimelineBoxFullscreen overlay, TraceTimelineOverlay
PurposeSummary and previewExplore, filter and investigate
Spans shownThe first 10 spans of the traceAll spans, as the full tree
FiltersNoneService, duration, search and breakdown
SidebarNoneAttributes sidebar
Row height28px40px: name and namespace on two lines
EntryEmbedded in other viewsThe preview button, or a click on a span

The preview hands the rest of the trace to the overlay with a "+N more" link.

Overlay layout

The overlay follows a fixed order from top to bottom. It mirrors the reader's path: choose what to look at, narrow it down, look, then look closer.

The spans column and the sidebar can both be resized, and their widths persist across sessions in localStorage.

PanelMinDefaultMaxStorage key
Spans column240460720trace-timeline-spans-column-width
Attributes sidebar320420800trace-timeline-attributes-sidebar-width

Show the resize handles only at the md breakpoint and up, and hide them on narrow screens. A handle turns blue-400 on hover and blue-500 while dragging.

Span row

One row is one span. The left half is identity: the span's position in the tree, and its name. The right half is timing: the bar and the duration.

The span tree. Rows are joined by tree connectors. Spans with children show a child count and a chevron, and leaf spans show a dot. A purple PublicApi badge with a dashed line marks where a service starts. Each row has a coloured bar and a duration label.

Always render a duration through FormattedDuration, so the switch between ns, µs, ms and s stays consistent.

Span groups and colour

Bar colour encodes the span's type, its group. The group is resolved in priority order, in spanGroups.js:

  1. The customer-set appsignal.group attribute.
  2. OpenTelemetry semantic attributes: db.system, messaging.system, rpc.system, or http.method with span_kind.
  3. Other.

Each group has its own colour token, from --span-web-server to --span-others. The values are on the semantic colour page.

The breakdown bar reuses this palette as a legend that is also a filter. Toggling a group changes how the bars of that colour appear in the timeline.

The breakdown bar. Under EVENTS are Errors, Queries and Events checkboxes, each with an icon. Under SPAN GROUP are Web Server, Other and Database / ORM checkboxes, each filled in its group's colour.

Event and issue markers

Extra signals on a span show as small icons only. The detail is in the sidebar, through AttributesPreview.

MarkerIconMeaning
Errorfa-bug, in the error colourThe span contains an exception event
Eventfa-calendarThe span contains an event that is not an exception
Queryfa-databaseThe span contains a database query
N+1fa-cloneThe span is an N+1 duplicate

A span with an exception is emphasised: its name and bar take the error colour, and when it is selected it gets the error background and border. Errors should be the first thing a reader spots on the timeline. The detail is in the sidebar's exception banner and its Exceptions tab.

Two rows from a timeline. The execute_query row has an error: its name, bug icon, bar and left accent are red, on a pale red background. The clickhouse.query row below it carries an amber x8 badge, marking an N+1 group of eight.

Filtering

Filtering has two layers. The toolbar narrows the result set, so spans that do not match disappear. The breakdown bar toggles groups on and off, so spans are emphasised or hidden.

In the breakdown bar:

Matches propagate from child to parent. If a child matches and its parent does not, the parent stays in the tree but is dimmed with opacity-25. Matching spans stand out without losing their place in the tree. This is done in nestSpans.

Row states

A row can be in several states at once, so the states have a precedence.

StateTriggerTreatment
SelectedA click. The span is shown in the sidebarA left accent bar and the surface-highlight background
HighlightedHoverThe surface-highlight background
ZebraEven rowsThe surface-raised background
DimmedA group hover, or a partial matchopacity-25
ExceptionThe span contains an exceptionError-coloured text and bar, and the error background when selected

Group emphasis on hover takes precedence over dimming from a partial match. The exception state replaces the selected and highlighted colours with the error colour.

Use the left accent bar for selection, so it is clear which span the sidebar is showing. Keep the hover treatment different, so a passing state never looks like a fixed one.

Moving the pointer across a row shows a vertical cursor guide line. It helps the reader see how bars line up on the time axis.

Attributes sidebar

The sidebar holds the full context of the selected span, or of the hovered one. A hovered span takes precedence over the selected one, so the reader gets a fast preview first.

The attributes sidebar for a POST /api/v2/metrics/timeseries span. The header has the namespace, the span name, badges for http.server and status:ok, the start time and duration, a memory allocation line, and a trace ID and span ID, each with a copy button. A query block follows, then Attributes, Events (3) and Exceptions (2) tabs over a two-column attribute grid.

Spans with an error

When a span has an exception, the sidebar switches to an error state. It gains a status:error badge, an exception banner with the count, the message and a View button, and an Exceptions tab. This is where an investigation that starts at an error marker in the timeline ends.

The sidebar in its error state. A red banner under the time reads "2 exceptions on this span" with the exception message and a View button. The Exceptions (2) tab is selected and shows Exception #1 with its name, incident digest, stacktrace ID, subroot ID and message.

Cross-service spans and span links

Where a trace crosses services, boundaries and links are marked consistently in purple.

Performance

N+1 groups

This section documents an in-progress design that is not in the code yet. The right-panel mockups below are recreated from the design screenshots — treat them as intent, not shipped UI.

When the same query repeats many times, an N+1, the timeline collapses the repetitions into one representative span and moves the details to the right panel. A trace with hundreds of duplicate queries stays readable.

The design follows three principles:

Representative span and count badge

Today's x8 badge, shown under the Event and issue markers section, is the same pattern at a small scale. The N+1 design adds expanding, hover highlighting and a summary panel.

N+1 summary panel

Clicking the representative span opens the N+1 Summary tab. It shows how many queries ran, their total duration, and what percentage of the trace they account for. A Spans card lists the slowest, mean and fastest occurrences. Clicking Slowest or Fastest jumps to that span in the timeline and highlights it.

A recreated mockup of the N+1 Summary tab for a group_selectors span badged x99. It shows QUERIES 99, TOTAL 24 ms and OF THE TRACE 66%. A Spans card lists the slowest at 1.42s, the mean of 99 spans at 233 µs and the fastest at 68 µs, each with a bar. A Nesting card follows.

Nested N+1

When repetitions are nested, each level shows its own count badge, and the summary adds a Nesting panel. The panel shows the chain of repeated spans, the cumulative multiplication, and a breakdown per level. Clicking the representative span shows the nesting: each span name with its repetition count, and the number of nesting levels.

A recreated mockup of the Nesting panel, titled "Nesting (3 levels)". It shows the chain SELECT tags ×3, SELECT tag_types ×2, SELECT locales ×4, and in purple "cumulative 99 × 3 × 2 × 4 = 8.5M queries". Below are rows for Level 1, 2 and 3, each with its query, count and duration.

Working inside an expanded group

After a group is expanded, clicking one of its spans shows that span's attributes like any other span. A "Part of an N+1 group" message stays fixed at the bottom of the panel, with a link back to the summary, so the way back is always available.