LangChain & LangGraph in one line

1. Install

pip install doairight-evals

The core client is standard-library only. The LangChain integration imports langchain_core, which you already have if you’re using LangChain — a plain import doairight_evals never pulls it in.

2. Attach the handler

from doairight_evals import create_client
from doairight_evals.integrations.langchain import DoAIRightCallbackHandler

client = create_client(api_key="dar_...")            # key: /app/developer
handler = DoAIRightCallbackHandler(client, metadata={"app": "my-app"})

# Per call:
llm.invoke(prompt, config={"callbacks": [handler]})

# ...or set it once so every call through this model is captured:
llm = ChatOpenAI(model="gpt-4o", callbacks=[handler])

Set it once on the model, chain, or agent and every call is captured — the callback list propagates down to nested calls automatically.

3. LangGraph

from doairight_evals.integrations.langgraph import instrument

# Wrap a compiled graph once — every node's LLM call is now observed.
graph = instrument(app.compile(), client, metadata={"app": "my-agent"})
graph.invoke({"input": "..."})   # graph API is unchanged

instrument() injects the handler into the graph’s callbacks and returns the same graph — its API is unchanged.

Add fields to every run

handler = DoAIRightCallbackHandler(
    client,
    kind="generative",              # DoAIRight run kind (default)
    metadata={"app": "my-app"},
    session_id="conversation-123",  # any extra fields are merged into every run
)

Any extra keyword argument is merged into every observe() the handler emits. Per-run values the handler derives — model, provider, latency, token counts — take precedence.

What it captures — and what it won’t break

• Streaming works the same. LangChain fires its end callback with the fully-assembled result even when tokens were streamed, so you get the complete response — no buffering code on your side.
• Chat and completion models, and every nested LLM call inside agents and graphs.
• Model, provider, latency, and token usage are extracted best-effort from the call.
• A failed model call is never recorded as a success — the handler drops it rather than inventing a response.
• It never throws into your app. Capture is fire-and-forget; errors are swallowed (optionally surfaced via the client’s on_error).

Not on LangChain, or using JS/TS?

The one-line callback handler is Python. For a direct provider SDK, a non-LangChain path, or a TypeScript/JS app, use the two-line manual call instead: Any app — direct SDK.