Trace

What it does

The trace package provides opt-in observability for Phero. A tracer receives typed Event values at each significant lifecycle point: agent start and end, loop iterations, LLM requests and responses, tool calls and results, and memory retrieval and persistence.

Built-in tracers

• trace.Noop: default zero-cost tracer that discards all events (top-level trace package)
• text.New(w) / text.NewDefault(): writes human-readable, colorized trace lines to any io.Writer; NewDefault() targets os.Stderr
• jsonfile.New(filePath): appends one JSON record per line (NDJSON) to a file; call Close() when done
• otel.New(tracer) / otel.NewDefault(): maps events to OpenTelemetry spans for export to any OTLP backend (Jaeger, Tempo, Honeycomb, …)

Import paths: github.com/henomis/phero/trace/text, github.com/henomis/phero/trace/jsonfile, and github.com/henomis/phero/trace/otel.

OpenTelemetry tracer

trace/otel turns an agent run into a tree of OpenTelemetry spans: the run is a root span, and each LLM call and tool call become child spans. Token usage, cost, model name, and tool arguments are attached as span attributes (using the OpenTelemetry GenAI semantic-convention names where applicable).

The package depends only on the OpenTelemetry API. Your application owns the SDK wiring — a TracerProvider and an exporter — so phero never pulls in a specific exporter. Install a global provider and use otel.NewDefault(), or pass an explicit tracer to otel.New(tracer).

import (
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/sdk/trace"

    pherotel "github.com/henomis/phero/trace/otel"
)

// Application wires the SDK + exporter (done once at startup).
tp := trace.NewTracerProvider(trace.WithBatcher(exporter))
otel.SetTracerProvider(tp)
defer tp.Shutdown(context.Background())

// Attach the OpenTelemetry tracer to an agent.
a.SetTracer(pherotel.NewDefault())

A single tracer instance correlates events by agent name (plus iteration or tool call ID), so sequential runs and handoffs to differently named agents are fully supported.

Attach tracing to an agent

Agents expose SetTracer. Once attached, every Run emits lifecycle events automatically.

import (
    "context"
    "fmt"
    "os"

    "github.com/henomis/phero/agent"
    "github.com/henomis/phero/llm"
    "github.com/henomis/phero/llm/openai"
    "github.com/henomis/phero/trace/text"
)

func main() {
    client := openai.New(os.Getenv("OPENAI_API_KEY"))

    a, err := agent.New(
        client,
        "math-agent",
        "You are a helpful math assistant.",
    )
    if err != nil {
        panic(err)
    }

    a.SetTracer(text.New(os.Stderr))

    result, err := a.Run(context.Background(), llm.Text("What is 12 * 9?"))
    if err != nil {
        panic(err)
    }

    fmt.Println(result.TextContent())
}

Trace raw LLM calls

If you want observability around direct llm.LLM.Execute calls, wrap any backend with trace.NewLLM.

tracedClient := trace.NewLLM(client, text.New(os.Stderr))

result, err := tracedClient.Execute(ctx, messages, tools)

When the wrapper runs inside an agent, request and response events are automatically annotated with the agent name and iteration number via context propagation.

JSON file tracer

For machine-readable output, use jsonfile.New to write NDJSON to a file. Each line is a self-contained JSON record with a type, timestamp, and data field.

import "github.com/henomis/phero/trace/jsonfile"

t, err := jsonfile.New("trace.ndjson")
if err != nil {
    panic(err)
}
defer t.Close()

a.SetTracer(t)

The tracer is goroutine-safe and appends to the file if it already exists.

Run summary

After every agent run, a AgentRunSummaryEvent is emitted with a RunSummary containing aggregated metrics. The same summary is also available on the Result returned by Run().

• RunSummary.Usage — UsageSummary{InputTokens, OutputTokens, CostUSD} — CostUSD is the best-effort total cost from per-model pricing (see llm.RegisterPricing)
• RunSummary.Latency — LatencySummary{Total, LLM, Tool, Memory}
• RunSummary.Tools — []ToolCallSummary{ToolName, Calls, Errors}
• RunSummary.Iterations, LLMCalls, ToolCalls, ToolErrors
• RunSummary.HandoffAgent — non-empty when the run ended via a handoff

result, err := a.Run(ctx, llm.Text(input))
if err != nil {
    panic(err)
}
if s := result.Summary; s != nil {
    fmt.Printf("iterations=%d llmCalls=%d tokens=%d+%d cost=$%.4f\n",
        s.Iterations, s.LLMCalls,
        s.Usage.InputTokens, s.Usage.OutputTokens, s.Usage.CostUSD)
    fmt.Printf("total=%s  llm=%s  tool=%s\n",
        s.Latency.Total, s.Latency.LLM, s.Latency.Tool)
}

Event types

• AgentStartEvent and AgentEndEvent
• AgentIterationEvent
• AgentRunSummaryEvent — emitted once per run with a full RunSummary
• LLMRequestEvent and LLMResponseEvent
• ReasoningEvent — emitted after a response that contains extended-thinking content
• ToolCallEvent and ToolResultEvent
• MemoryRetrieveEvent and MemorySaveEvent

Implement the trace.Tracer interface if you want to send these events somewhere other than the terminal.

Context propagation

The tracer is carried through the request context with trace.WithTracer and read back with trace.FromContext. This lets tool handlers and lower-level helpers emit their own trace events without depending directly on the agent type.

Try it

The tracing example shows a calculator tool plus a traced agent run from the terminal.

# from repo root
go run ./examples/tracing

Related packages

• agent: emits trace events during orchestration
• llm: backends can be wrapped with trace.NewLLM
• memory: memory retrieval and save operations are traced when used through agents