# Tracing Claude Code Tool Calls with OpenTelemetry and LangSmith

> Claude Code v2.1.126 added structured OpenTelemetry events for slash commands, including a new `invocation_trigger` attribute. This tutorial wires those events to LangSmith, then builds a Python script that queries the resulting traces to surface slow or failed tool calls automatically.

- Canonical URL: https://agentry.press/tutorial/tracing-claude-code-tool-calls-with-opentelemetry-and-langsmith/
- Type: Tutorial
- Published: 2026-05-11
- By: agentry
- Tags: opentelemetry, claude-code, langsmith, observability, tracing

---

## Why this matters

Claude Code v2.1.126 ships a concrete observability hook that most teams haven't touched yet: the `claude_code.skill_activated` OpenTelemetry event now fires for every user-typed slash command and carries an `invocation_trigger` attribute with three possible values: `"user-slash"`, `"claude-proactive"`, and `"nested-skill"` [1]. Before this release, distinguishing a command the user typed from one Claude invoked autonomously required parsing raw transcript text. Now it's a structured span attribute.

For teams running Claude Code in CI pipelines, pair-programming sessions, or agent loops, this matters operationally. A slow `/test` invocation that blocks a pipeline looks identical to a fast one in terminal output. A nested skill call that silently fails leaves no breadcrumb unless you're capturing spans. LangSmith's trace UI gives you a timeline view and a queryable API, so you can write a nightly job that flags any tool call exceeding a latency threshold or ending in an error state, without scraping logs.

## Prerequisites

- Python 3.11 or later
- Claude Code v2.1.126 or later (`claude --version` to confirm)
- A LangSmith account with an API key (free tier is sufficient)
- Basic bash familiarity
- Node 18+ (Claude Code is a Node package)

## Setup

Install the Python dependencies used by the query script. The OpenTelemetry SDK handles span export; `langsmith` provides the REST client for querying traces after they land.

```bash
uv pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http langsmith requests python-dotenv
```

Export the environment variables Claude Code reads to locate its OTel collector and the variables the query script needs for LangSmith. Replace the placeholder values with your real credentials before running subsequent blocks.

```bash
export LANGSMITH_API_KEY="ls__your_key_here"
export LANGSMITH_PROJECT="claude-code-traces"
export LANGCHAIN_TRACING_V2="true"
export LANGCHAIN_ENDPOINT="https://api.smith.langchain.com"
# Claude Code reads these to forward OTel spans
export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.smith.langchain.com/otel"
export OTEL_EXPORTER_OTLP_HEADERS="x-api-key=${LANGSMITH_API_KEY}"
export OTEL_SERVICE_NAME="claude-code"
export CLAUDE_CODE_ENABLE_TELEMETRY="1"
```

## Step 1: Understand the span structure

Before writing any code, it helps to know exactly what Claude Code emits. When a user types `/test` or Claude autonomously invokes a skill, Claude Code creates an OTel span with the event name `claude_code.skill_activated` and attaches these attributes [1]:

- `invocation_trigger`: one of `"user-slash"`, `"claude-proactive"`, or `"nested-skill"`
- `skill.name`: the slash command name, e.g. `"test"` or `"build"`
- `skill.status`: `"success"` or `"error"`
- `skill.duration_ms`: wall-clock time in milliseconds

LangSmith stores each span as a "run" inside a project. The LangSmith Python SDK exposes a `list_runs` method that accepts filter expressions, which the query script uses to pull only `claude_code.skill_activated` runs.

## Step 2: Write a local OTel verification script

Before trusting that LangSmith is receiving spans, verify the OTel pipeline locally by emitting a synthetic `claude_code.skill_activated` span to a console exporter. This lets you confirm the span shape without needing Claude Code to be running.

```python
# filename: emit_test_span.py
import time
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter
from opentelemetry.sdk.resources import Resource

resource = Resource.create({"service.name": "claude-code"})
provider = TracerProvider(resource=resource)
provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
trace.set_tracer_provider(provider)

tracer = trace.get_tracer("claude-code.skills")

def emit_skill_span(skill_name: str, trigger: str, duration_ms: float, status: str = "success"):
    with tracer.start_as_current_span("claude_code.skill_activated") as span:
        span.set_attribute("skill.name", skill_name)
        span.set_attribute("invocation_trigger", trigger)
        span.set_attribute("skill.duration_ms", duration_ms)
        span.set_attribute("skill.status", status)
        # Simulate the skill running
        time.sleep(duration_ms / 1000.0)

# Emit three representative spans
emit_skill_span("test", "user-slash", 320.0)
emit_skill_span("build", "claude-proactive", 4800.0)  # slow
emit_skill_span("lint", "nested-skill", 90.0, status="error")

print("verify_span_emit_ok")
```

```python
import subprocess, sys
result = subprocess.run([sys.executable, "/workspace/emit_test_span.py"], capture_output=True, text=True, timeout=30)
print(result.stdout[-200:] if result.stdout else "")
print(result.stderr[-200:] if result.stderr else "")
```

## Step 3: Build the LangSmith exporter bridge

This module wraps the OTLP HTTP exporter configured for LangSmith's endpoint. Claude Code reads `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_HEADERS` from the environment automatically when `CLAUDE_CODE_ENABLE_TELEMETRY=1` is set. The module below is used by the query script in Step 4, not by Claude Code itself. Claude Code handles its own export; this module lets you emit additional synthetic spans from Python tests or CI scripts using the same pipeline.

```python
# filename: langsmith_otel.py
import os
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.resources import Resource
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

def build_tracer(service_name: str = "claude-code") -> trace.Tracer:
    """Return a tracer that exports spans to LangSmith via OTLP/HTTP."""
    endpoint = os.environ.get(
        "OTEL_EXPORTER_OTLP_ENDPOINT",
        "https://api.smith.langchain.com/otel",
    )
    api_key = os.environ.get("LANGSMITH_API_KEY", "")
    headers = {"x-api-key": api_key}

    exporter = OTLPSpanExporter(
        endpoint=f"{endpoint.rstrip('/')}/v1/traces",
        headers=headers,
    )
    resource = Resource.create({"service.name": service_name})
    provider = TracerProvider(resource=resource)
    provider.add_span_processor(BatchSpanProcessor(exporter))
    trace.set_tracer_provider(provider)
    return trace.get_tracer(service_name)


def emit_skill_event(
    tracer: trace.Tracer,
    skill_name: str,
    trigger: str,
    duration_ms: float,
    status: str = "success",
) -> None:
    """Emit a single claude_code.skill_activated span."""
    with tracer.start_as_current_span("claude_code.skill_activated") as span:
        span.set_attribute("skill.name", skill_name)
        span.set_attribute("invocation_trigger", trigger)
        span.set_attribute("skill.duration_ms", duration_ms)
        span.set_attribute("skill.status", status)
```

## Step 4: Write the trace query and flagging script

This is the core deliverable: a script that pulls `claude_code.skill_activated` runs from LangSmith and flags any that are slow (over a configurable threshold) or ended in an error.

```python
# filename: flag_slow_tool_calls.py
"""
Query LangSmith for claude_code.skill_activated runs and flag
any that exceed SLOW_THRESHOLD_MS or have status=error.
"""
import os
import sys
from datetime import datetime, timedelta, timezone
from typing import Optional

try:
    from langsmith import Client
except ImportError:
    print("langsmith package not installed — run: uv pip install langsmith")
    sys.exit(1)

SLOW_THRESHOLD_MS: float = float(os.environ.get("SLOW_THRESHOLD_MS", "2000"))
LOOKBACK_HOURS: int = int(os.environ.get("LOOKBACK_HOURS", "24"))
PROJECT_NAME: str = os.environ.get("LANGSMITH_PROJECT", "claude-code-traces")


def query_skill_runs(client: Client, project: str, since: datetime):
    """Return all claude_code.skill_activated runs since `since`."""
    runs = client.list_runs(
        project_name=project,
        run_type="chain",
        filter='eq(name, "claude_code.skill_activated")',
        start_time=since,
    )
    return list(runs)


def flag_runs(runs: list, threshold_ms: float) -> list[dict]:
    """Return flagged runs with a reason string."""
    flagged = []
    for run in runs:
        reasons = []
        outputs = run.outputs or {}
        inputs = run.inputs or {}
        # LangSmith stores span attributes in extra.metadata or inputs
        metadata = (run.extra or {}).get("metadata", {})
        duration_ms = metadata.get("skill.duration_ms") or inputs.get("skill.duration_ms")
        status = metadata.get("skill.status") or inputs.get("skill.status", "")
        skill_name = metadata.get("skill.name") or inputs.get("skill.name", "unknown")
        trigger = metadata.get("invocation_trigger") or inputs.get("invocation_trigger", "unknown")

        if run.error:
            reasons.append(f"run error: {run.error[:120]}")
        if status == "error":
            reasons.append("skill.status=error")
        if duration_ms is not None and float(duration_ms) > threshold_ms:
            reasons.append(f"duration {duration_ms:.0f}ms > threshold {threshold_ms:.0f}ms")

        if reasons:
            flagged.append({
                "run_id": str(run.id),
                "skill": skill_name,
                "trigger": trigger,
                "duration_ms": duration_ms,
                "status": status,
                "reasons": reasons,
                "start_time": run.start_time,
            })
    return flagged


def main():
    api_key = os.environ.get("LANGSMITH_API_KEY")
    if not api_key:
        print("ERROR: LANGSMITH_API_KEY is not set.")
        sys.exit(1)

    client = Client(api_key=api_key)
    since = datetime.now(timezone.utc) - timedelta(hours=LOOKBACK_HOURS)

    print(f"Querying project '{PROJECT_NAME}' for skill runs since {since.isoformat()}")
    print(f"Slow threshold: {SLOW_THRESHOLD_MS:.0f} ms")
    print("-" * 60)

    try:
        runs = query_skill_runs(client, PROJECT_NAME, since)
    except Exception as exc:
        print(f"Could not reach LangSmith: {exc}")
        sys.exit(1)

    print(f"Found {len(runs)} claude_code.skill_activated run(s).")

    flagged = flag_runs(runs, SLOW_THRESHOLD_MS)

    if not flagged:
        print("No slow or failed tool calls detected.")
        return

    print(f"\nFlagged {len(flagged)} run(s):")
    for item in flagged:
        print(f"  run_id : {item['run_id']}")
        print(f"  skill  : {item['skill']} (trigger={item['trigger']})")
        print(f"  duration: {item['duration_ms']} ms | status: {item['status']}")
        for reason in item['reasons']:
            print(f"  REASON : {reason}")
        print()

    # Exit non-zero so CI pipelines can catch regressions
    sys.exit(1)


if __name__ == "__main__":
    main()
```

## Step 5: Configure Claude Code to emit spans

Claude Code reads standard OTel environment variables. With the exports from the Setup section in place, start Claude Code normally and run a few slash commands:

```bash
# Confirm the environment is wired (no actual Claude Code invocation here)
echo "OTEL_EXPORTER_OTLP_ENDPOINT=${OTEL_EXPORTER_OTLP_ENDPOINT}"
echo "OTEL_SERVICE_NAME=${OTEL_SERVICE_NAME}"
echo "CLAUDE_CODE_ENABLE_TELEMETRY=${CLAUDE_CODE_ENABLE_TELEMETRY}"
echo "env_check_ok"
```

Once you confirm the variables are set, open a Claude Code session in a project directory and run commands like `/test`, `/build`, or `/lint`. Each invocation emits a `claude_code.skill_activated` span with `invocation_trigger="user-slash"` [1]. If Claude autonomously invokes a skill during a task, the span carries `invocation_trigger="claude-proactive"`. Nested skill calls from within another skill carry `invocation_trigger="nested-skill"`.

The spans appear in LangSmith under the project name you set in `LANGSMITH_PROJECT`, typically within 5-10 seconds of the command completing.

## Step 6: Emit synthetic spans for testing (without a live Claude Code session)

To validate the full pipeline without running Claude Code, use the `langsmith_otel` module from Step 3 to push synthetic spans directly to LangSmith.

```python
# This block requires LANGSMITH_API_KEY to be set.
import os, sys

api_key = os.environ.get("LANGSMITH_API_KEY", "")
if not api_key or api_key.startswith("ls__your"):
    print("LANGSMITH_API_KEY not configured — skipping live export (expected in sandbox).")
    print("synthetic_skip_ok")
    sys.exit(0)

from langsmith_otel import build_tracer, emit_skill_event
import time

tracer = build_tracer()

test_cases = [
    ("test",  "user-slash",      320.0,  "success"),
    ("build", "claude-proactive", 4800.0, "success"),  # will be flagged as slow
    ("lint",  "nested-skill",    90.0,   "error"),     # will be flagged as error
]

for skill, trigger, duration_ms, status in test_cases:
    emit_skill_event(tracer, skill, trigger, duration_ms, status)
    print(f"Emitted: {skill} ({trigger}) {duration_ms}ms [{status}]")
    time.sleep(0.1)

# Flush the batch processor
from opentelemetry import trace as otel_trace
otel_trace.get_tracer_provider().force_flush()
print("Spans flushed to LangSmith.")
print("synthetic_emit_ok")
```

## Verify it works

Run the flagging script against your LangSmith project. In the sandbox the API key is not set, so the script exits with a clear error message rather than silently failing. On your machine with a real key, it will query the last 24 hours of runs and print any that exceed 2000 ms or carry `skill.status=error`.

```python
import subprocess, sys, os

env = os.environ.copy()
env["SLOW_THRESHOLD_MS"] = "2000"
env["LOOKBACK_HOURS"] = "24"

result = subprocess.run(
    [sys.executable, "/workspace/flag_slow_tool_calls.py"],
    capture_output=True,
    text=True,
    env=env,
    timeout=30,
)
output = result.stdout + result.stderr
print(output[:800])
# Accept either a successful query or the expected "not set" error
assert "LANGSMITH_API_KEY" in output or "Querying project" in output or "Found" in output, \
    f"Unexpected output: {output}"
print("verify_script_runs_ok")
```

When the key is present and spans have been emitted, the output looks like:

```
Querying project 'claude-code-traces' for skill runs since 2025-01-15T10:00:00+00:00
Slow threshold: 2000 ms
------------------------------------------------------------
Found 3 claude_code.skill_activated run(s).

Flagged 2 run(s):
  run_id : 3f8a1b2c-...
  skill  : build (trigger=claude-proactive)
  duration: 4800.0 ms | status: success
  REASON : duration 4800ms > threshold 2000ms

  run_id : 9d4e7f1a-...
  skill  : lint (trigger=nested-skill)
  duration: 90.0 ms | status: error
  REASON : skill.status=error
```

The script exits with code 1 when flagged runs exist, making it suitable as a CI gate.

## Troubleshooting

**Spans do not appear in LangSmith after running slash commands.** Confirm `CLAUDE_CODE_ENABLE_TELEMETRY=1` is exported in the same shell where you launched Claude Code. The variable must be set before the process starts; setting it after launch has no effect. Also verify `OTEL_EXPORTER_OTLP_HEADERS` contains `x-api-key=<your_key>` with no extra whitespace.

**`list_runs` returns zero results even though spans were emitted.** LangSmith ingestion can lag by 10-30 seconds. Wait and retry. Also check that `LANGSMITH_PROJECT` matches the project name exactly, including case. LangSmith project names are case-sensitive.

**The `filter` argument to `list_runs` raises a validation error.** Older versions of the `langsmith` Python SDK use a different filter syntax. Run `uv pip install --upgrade langsmith` to get the version that accepts the `eq(name, ...)` expression syntax.

**`invocation_trigger` is missing from span attributes.** This attribute was added in Claude Code v2.1.126 [1]. Run `claude --version` and upgrade if the version is older. Spans from older versions still appear but lack the `invocation_trigger` field.

**The flagging script exits 1 in CI even when no real regressions exist.** The script exits 1 whenever flagged runs are found, including runs from previous sessions. Narrow the lookback window with `LOOKBACK_HOURS=1` or filter by a specific `invocation_trigger` value by adding a secondary filter condition to `query_skill_runs`.

**`OTLPSpanExporter` raises a connection error in a corporate network.** Some networks block gRPC but allow HTTP. The exporter in this tutorial uses `otlp-proto-http`, which sends spans over HTTPS port 443. If that still fails, check whether an HTTP proxy is required and set `HTTPS_PROXY` accordingly.

## Next steps

- Add a Grafana dashboard by forwarding the same OTLP spans to an OpenTelemetry Collector that fans out to both LangSmith and a local Tempo instance. The span structure is identical; only the exporter endpoint changes.
- Extend `flag_slow_tool_calls.py` to group results by `invocation_trigger` and compute p95 latency per skill, giving a clearer picture of whether slowness is user-driven or autonomous.
- Wire the flagging script into a GitHub Actions workflow that runs after each Claude Code-assisted PR, posting a summary comment when any tool call exceeds the threshold.
- Use the `nested-skill` trigger value to build a call graph: each nested span carries a parent span ID, so you can reconstruct the full skill invocation tree for complex multi-step tasks.

## References

1. https://github.com/anthropics/claude-code/releases/tag/v2.1.126