Traces & observability

Every One API response is traceable. This chapter covers the trace id, the GET /v1/traces/:id read, execution history and cost, the response headers, and what is not observable through the API today. Base URL is https://api.plungeai.com; reads use an ozk_ key (Authentication). Source: orchestration/api-gateway/routes/traces.ts, orchestration/api-gateway/lib/execution-row.ts.

12.1 The trace id

Every response carries x-request-id, the router's id for that one request (on the models plane the inference gateway's, 13.8). A run's trace id is its run id: workflow_id / execution_id in the body, x-execution-id where the response sets it, and workflow_id on every SSE frame. Without x-trace-id the router uses its request id as the trace id, so x-request-id, x-execution-id, workflow_id and the body's request_id are one UUID (live-verified 2026-09-18 for one agent execute) and you can trace a run from any of them.

You may also supply the trace id: send x-trace-id: <uuid> and the run adopts it (a replayed UUID that already ran → 409 duplicate_execution_id, see Errors). x-request-id and the body's request_id then stay the router's per-request id: read the trace by the id you sent (it is also the run's workflow_id), not by request_id, which answers 404 (live-verified 2026-10-04: GET /v1/traces/<x-trace-id> → 200, GET /v1/traces/<x-request-id> → 404).

12.2 Response headers

The execution planes emit no x-ocean-* headers; the models plane adds x-ocean-router-id, x-ocean-cost-usd and x-ocean-dropped-params. Every header and the eleven a browser may read (CORS expose list) are in 13.8. Rate-limit state is not returned as a header — read it with plungeai_whoami (the MCP plane) or see Authentication.

12.3 GET /v1/traces/:id

Reads one trace by id and joins two telemetry tables (both on ocean-telemetry via TELEMETRY_DB, per the root CLAUDE.md DB map):

{ "trace_id": "<id>", "spans": [ … ], "gateway_requests": [ … ] }

• spans — rows of execution_trace_spans (columns: id, trace_id, workflow_id, task_id, ts, type, agent, status, duration_ms, payload_ref, payload_inline), ordered by time, up to 500. These are written by trace-service off the async trace-events queue, so they can lag a just-finished run or be empty on a short one.
• gateway_requests — rows of api_gateway_requests (columns: id, trace_id, plane, route, user_id, status, upstream, duration_ms, cost_usd, created_at): the router's own request log — one row per API call on this trace, including the plane, route, HTTP status and per-request cost.

An unknown trace id — and a trace that belongs to another tenant — is 404 not_found, the same answer for both, so a foreign id is never confirmed. You own a trace when your key started the run (its execution row is written synchronously, before the engine dispatch) or made the first request logged under it (tool calls). A run that just finished is therefore 200 even while its spans are still empty — those are queue-written and can lag. A request's own log row lands a moment after its response: a 404 for an id you have only just used means "nothing recorded yet", so retry in a few seconds.

Execute without x-trace-id, then read the trace by its request_id (here it is the trace id):

RESP=$(curl -s "https://api.plungeai.com/v1/agents/one-agent/execute" \
  -H "Authorization: Bearer $PLUNGE_API_KEY" -H 'Content-Type: application/json' \
  -d '{"prompt":"Reply with only: pong","max_tokens":24}')
RID=$(printf '%s' "$RESP" | sed -n 's/.*"request_id":"\([^"]*\)".*/\1/p')
echo "trace=$RID"
sleep 3
TRACE=$(curl -s "https://api.plungeai.com/v1/traces/$RID" \
  -H "Authorization: Bearer $PLUNGE_API_KEY")
echo "$TRACE"
echo "$TRACE" | grep -q "\"trace_id\":\"$RID\"" && echo TRACE_MATCH_OK

An unknown trace id — 404 not_found:

curl -s "https://api.plungeai.com/v1/traces/00000000-0000-4000-8000-000000000006" \
  -H "Authorization: Bearer $PLUNGE_API_KEY"

12.4 Execution history & cost

Every agent and workflow run opens a workflow_executions row on ocean-db (DB), source: 'api' (lib/execution-row.ts). The row carries status (running → completed / failed), input, workflow_name, duration, end_time, execution_summary (the per-task durations the pricer reads) and, once priced, the run cost. It is closed from the engine's response after the run; a run the platform cannot record does not run "for free".

Where a developer sees these rows:

• MCP plane — plungeai_executions (list / get / output / conversation) and plungeai_get_result return your run history and results as user-ready markdown (MCP plane).
• Studio — your runs appear in the run history and per-run views.
• Dashboard — dashboard.plungeai.com surfaces run status and cost roll-ups.

Cost is computed asynchronously by the workflow-cost worker from execution_summary; a run whose summary was preserved is priced (an empty-completion or retrieval fault is flipped to failed without wiping the summary, so the reasoning tokens it spent are still metered).

12.5 What is not observable via the API today

• There is no public per-token or per-span streaming metrics endpoint; live progress comes from the SSE streams (Streaming, and the workflows event vocabulary in Workflows plane).
• GET /v1/traces/:id spans depend on async queue processing — treat an empty spans array as "not landed yet", and use gateway_requests (written inline) plus the execution row for the authoritative status.
• There is no dedicated cost endpoint on this plane — read cost from the execution row via MCP / Studio / Dashboard (12.4).
• You read only your own traces; you cannot read another tenant's trace.