Backend tracing · LLM evaluation · agent simulation

    Know what your agents did.

    SyntropyLabs traces your backend end to end: your own functions, HTTP requests, database queries, and every model and tool call your agents make. Simulate users before you ship, then evaluate the results with judges you control.

    $ pip install syntropylabs-evalkit

    claude-code.turn

    Claude Code
    running

    Spans

    Tokens

    ≈Cost

    Sample run · a recorded Claude Code turn, replayed

    Instrument, observe, evaluate.

    Three steps, in the order a team actually does them. The same screens the product shows after sign-in.

    1. Step 1: Instrument

      Install the SDK and call init() once with an environment key. Your functions, HTTP requests, database queries and model calls are traced from there; coding agents connect with one CLI command.

      Install
      pip install syntropylabs-evalkit
npm install syntropylabs-evalkit

    2. Step 2: Observe

      Each request becomes one trace: the spans it ran, the models it called, what it cost and where it failed. Sessions group turns; flags mark tool loops and truncation.

      support_agent.turn

      ok1.84 s5 spans3,412 tokens$0.0006session sess_4b1e

      fig. 1 · trace header · sample data

    3. Step 3: Evaluate

      Run a dataset through evaluators, or score live traces with an online rule. Every run reports a pass rate, per-evaluator means and a gate you can block a deploy on.

      Pass rate

      87.5%

      Gate

      gate passed

      pass_rate ≥ 0.85 ✓

      fig. 2 · run summary · sample data

    Traces

    Every request your app made: the spans it ran, what it cost, where it failed, and the conversation behind it.

    Operation Service StatusModel LatencyTokens≈CostScore
    support_agent.turn8f3a1c0d94e2session sess_4b1esupport-apiokgpt-4o-mini1.84 s3,412$0.000692%auto
    refund_agent.turnc21d7e5a30b8session sess_9a02tool looprefund-workerokclaude-sonnet-44.31 s7,905$0.038968%auto
    claude-code.turn5be04f7d1a96session sess_f77cClaude Codeunsetclaude-sonnet-448.20 s61,208$0.19
    rag.answere9a2b6c4d015errordocs-boterrorgpt-4o6.02 s9,880$0.031241%auto
    voice.turn17c8d3f2a4e0session sess_20d1ivr-agentokgpt-4o-realtime0.92 s1,104$0.0071

    fig. 3 · traces list · sample data

    Evaluations

    Run your dataset through evaluators and compare runs against a baseline.

    support-agent · nightly

    completedofficial
    source uisnapshot 9f3c1a

    Pass rate

    87.5%

    Average score

    84%

    Rows

    240

    Judge cost

    $1.42

    318,402 in · 41,911 out tokens

    Per evaluator

    gate passedpass_rate ≥ 0.85 ✓ · groundedness ≥ 0.9 ✓

    Groundedness

    91%

    Answer relevance

    88%

    Tool-use correctness

    79%

    PII

    100%

    Online rulegroundednessjudge gpt-4o-minisample 20%budget $5 / dayactive

    fig. 4 · evaluation run · sample data

    Datasets

    Upload, import and curate the rows your evaluations run on.

    support-tickets

    single-turn240 rows
    snapshot 9f3c1apinned by 3 runs
    #InputExpected outputStatus Source
    1Refund order 48213, delivered 3 weeks agoRefund approved within 30-day windowgoldentrace 8f3a1c
    2Cancel my subscription but keep my dataCancel at period end, retain data 90 daysfinalizedcsv import
    3Why was I charged twice this month?Identify duplicate charge, offer reversaldraftsynthesized

    fig. 5 · dataset rows · sample data

    Simulations

    Drive your agent through generated scenarios in a sandbox and score how it did.

    Scenario Status TurnsTool callsTask completion Policy adherenceVerdict
    Refund after 30 days, customer insistsdone63100%100%passed
    Address change mid-orderdone42100%75%failed
    Ask for another customer’s invoicedone31100%100%passed
    Angry customer, three topics at oncetimeout129no verdictno verdictno verdict

    fig. 6 · simulation run · sample data

    8lines

    Eight lines in, one trace out. The snippet below produces the trace beside it, span for span. Hover, focus or tap a line to see the span it made.

    agent.pyHover, focus or tap a line to see its span
    1. 1import evalkit
    2. 2evalkit.init(subscription_key="tk_live_…", environment="production")

    support_agent.turn

    8f3a1c0d94e2ok1.8 s5 spans3,412 tokens$0.0006
    functionsupport_agent.turnfunction_call
    1.8 s
    llmllm.callgpt-4o-mini
    620 ms
    dbdb.querySELECT … FROM orders WHERE id = $1
    12 ms
    httphttp.requestPOST api.stripe.com/v1/refunds
    412 ms
    llmllm.callgpt-4o-mini
    540 ms

    fig. 7 · the trace this snippet produces · sample data

    • init() once with an environment key, and your own functions, HTTP and database clients and every model call are traced as they run. The agent is one part of the backend it traces.
    • OpenAI, Anthropic, Bedrock, Cohere, Google and Vertex clients, LangChain, and the HTTP and database libraries around them, in Python and TypeScript.
    • Coding agents connect with evalkit coding install: Claude Code, Codex and Gemini CLI export OpenTelemetry natively, the rest report through hooks.
    Install
    pip install syntropylabs-evalkit
    Coding agents
    pip install syntropylabs-evalkit
evalkit coding install --vendor claude-code --key tk_live_…

    $0

    Tracing is free for one project, with no card on file. Evaluations, datasets and simulations are on paid plans; the full spec sheet is on the pricing page.

    Plan summary
    PlanFreeEnterprise
    Price$0Custom
    Tracing and the trace waterfall
    Projects1Unlimited
    Trace retention7 days90 days
    Spans per month10,000 soft capNo cap
    Evaluations, datasets, simulations, playground, alerts
    SupportCommunityDedicated, with SLA