Known issues

What you see. A session in Monitor → Sessions with the conversation turns and nothing between them. Span count equals user turns plus agent turns. No waterfall.

Why. The SDK only listens for OpenTelemetry spans; something has to emit them. Either the framework’s instrumentation package is not installed, it was installed after redline dev started, or the agent is a hand-rolled loop that emits nothing by itself.

Fix.

• Pydantic AI — nothing to install; check redline dev is the venv’s copy.
• LangChain — pip install openinference-instrumentation-langchain, restart.
• CrewAI — openinference-instrumentation-crewai and openinference-instrumentation-openai, restart.
• Agno — openinference-instrumentation-agno, restart.
• Vercel AI SDK ≤6 — experimental_telemetry: { isEnabled: true } on the agent or each call. v7 needs nothing.
• Your own loop — emit two spans yourself: one per model call, one per tool call, in the gen_ai.* convention. See own tool loop §4.

How to confirm. A healthy session has a model call row with a token count under every agent turn. A CrewAI session went from 5 spans / 0 tools to 38 spans / 6 tools with nothing changed but the two packages.

What you see. “Where the time went” lists turn 1, turn 2, … turn 18 for what was one user message. The transcript on the right groups correctly under one “Worked for 12s”.

Why. The waterfall reads the span tree, and a bar with no parent is a root; each root is a turn. Spans emitted flat — no parent — are read as one turn each. Happens to hand-rolled loops using start_span() without parenting; the patched frameworks emit a proper tree.

Fix. Open one span per user turn and parent every model and tool span to it explicitly: start_span(name, context=trace.set_span_in_context(turn)). Do not make the turn span current with start_as_current_span inside a loop that yields — a current span leaks across yield. The turn span carries no gen_ai.* attribute, or it renders as an empty model call. Worked example on the own tool loop page.

What you see. Experiments → the run shows thinking and every tool call. Monitor → Sessions for the same conversation shows none.

Why. Two records. ctx.thinking / ctx.tool / ctx.tool_result write to the run. A session is filled only by conversation turns and OpenTelemetry spans. Both are correct; they are different things.

Fix. Nothing is broken. For the session to carry tool calls, spans must be emitted — see the first entry.

What you see. Every experiment run lands. Dev conversations land. Real user traffic produces no sessions at all.

Why. The @observe handler in agents.py is defined but nothing calls it. A decorator records calls to the function it decorates; the repository’s existing message pipeline — a webhook, a queue worker, a websocket loop — still runs its own code path.

Fix. The pipeline must call handle_message for each message. One call-site change, the only one the integration asks for in existing code. If the pipeline read anything off the agent’s stream besides the reply (an “ended” flag, a step count), pass a state dict the handler fills so nothing downstream changes. See monitoring — wiring it into an existing app.

What you see. The output row reads like <async_generator object …>, or the session opens with the input and never gets an answer.

Why. @observe was put on an async generator. It checks inspect.iscoroutinefunction, which is False for one, so it takes the sync path and records the generator object as the return value.

Fix. Decorate a coroutine that drains the stream and returns the text. Streaming entry points get a thin wrapper; the generator itself is untouched.

What you see. The task asks for something only machine_run can do and the agent replies “I have no way to do that”.

Why. Shape C without §4 — the doorway never merged the attached tools — or §5a missing: the tools are registered but the prompt never says they exist. On a patched framework, the agent was constructed before redline dev could patch it (a module that builds the agent on import, imported by something other than agents.py).

Fix. Shape C: redline_tools(task), as_openai_schema() to advertise, attached.call() to run, and a brief generated from attached.list in the prompt. Patched frameworks: run redline dev from a clean process and let agents.py be what imports the agent. Verify by counting tools.

Why. asyncio.run() in the doorway. It builds and destroys a loop per run; a framework that keeps async resources alive between calls (Agno’s pools, knowledge stores) then tears down into a closed loop.

Fix. async def run(task, ctx): return await _run(task, ctx). The SDK runs it on a persistent loop it owns.

What you see. The run’s error is a traceback ending in KeyError: 'SOME_CONFIG' from agents.py.

Why. The doorway needed something the repository knows and the SDK does not — which configured agent to run against, a tenant id — and it was never set on the machine running redline dev.

Fix. Put it in .env at the repository root; redline dev loads every variable in that file, not only REDLINE_*. Restart redline dev. Make the doorway raise a clear RuntimeError naming the variable rather than a bare KeyError, so the next person reads the reason in the run.

What you see. Two consecutive User rows; the second reads like the assistant (“Sure, I’d be happy to help…”). The agent’s real first reply appears one turn later.

Why. The doorway returned an empty string on that turn — typically the stream ended without a final event, so the doorway’s answer was never set. The worker skips empty text, so no agent row is written; the simulated user, handed a transcript with nothing back, wrote the assistant’s line itself.

Fix. Check the run’s own transcript for what turn one emitted. If the agent’s first turn legitimately produces no text (an onboarding step that only sets state), have the doorway return a placeholder so the turn is recorded; otherwise fix the stream mapping so final is captured.

Why. recursion_limit defaults to 25 steps. A real task needs a few hundred.

Fix. config={"recursion_limit": 250} on invoke / stream.

Why. The graph’s middleware implements only the sync wrap_model_call; LangChain refuses astream() over sync-only middleware.

Fix. Stream synchronously — agent.stream(...) in a plain def doorway.

Why. A tool raised and the exception propagated. LangGraph’s default ToolNode handler re-raises.

Fix. Make an exception the observation text the agent reads and recovers from: handle_tool_errors=True in LangGraph; a try/except returning the error string in your own tools. attached.call() already does this for attached tools.

no agents found — agents.py (or redline/*.ts) exports no @agent-decorated function / defineAgent object. Check the file name: agents.py, never redline.py, which shadows the SDK.

Registration rejected — the id collides with another agent in the project. Pick another kebab-case slug.

Why. Enforcement patches the openai and litellm clients. An agent that calls the provider over raw httpx or requests uses neither, so the patch sees nothing. Policies still arm in the redline dev process and govern any call that goes through a patched client.

Fix. There is no drop-in for a raw-HTTP loop. The gate calls exist in the SDK — policy.gate_tool_call, guard.judge, honeypot.armed() — and where they sit in your loop is a design decision; see own tool loop §5.

What you see. The Protect row reads waiting for first snapshot, the agent’s page reads No snapshots from this agent yet, and it has run — once, or a hundred times. The row may also be labelled with a framework the agent does not use.

Why. Nothing was ever captured, so nothing was ever sent. Snapshot capture reads a framework — a Pydantic AI / LangChain constructor, an Agno or CrewAI object in your module, the tools=[…] handed to a patched openai/litellm client. A loop of your own — a registry you wrote, the model over raw httpx — goes through none of them. Running it does not help, because for this shape capture is not run-triggered at all. The label comes from what is importable beside the agent: a venv carrying langchain for a retriever labels a hand-rolled loop langchain.

Fix. Declare the assets from the doorway, once the agent is built, with the experiment’s attached tools filtered out: declare_assets(system_prompt=…, tools=own_tools) (Python, SDK ≥ 0.2.27) or declareAssets({ systemPrompt, tools }) (TypeScript, SDK ≥ 0.2.20). The snapshot rides the next keepalive. Name the framework with @agent(..., framework="custom") so the row stops promising a capture that will not come. Worked example on the own tool loop page.

What you see. You ran one conversation. Protect shows a baseline and a candidate captured seconds apart — prompt changed · +1 tool — and a scan of each.

Why. run() is called once per turn, not once per conversation, so whatever the doorway does happens per message. Two things then went wrong at once: the doorway declared the agent’s tool list after the experiment’s attached tools had been merged into a reused session (so turn two declared one tool more than turn one), and — on SDKs before 0.2.27 — declare_assets accumulated, so a prompt that differs per turn was appended to the previous one until the snapshot hit its size ceiling. Each new hash is a new version, and every version is scanned.

Fix. Upgrade to SDK ≥ 0.2.27 / ≥ 0.2.20, where declaring replaces, and filter the attached tools out of what you declare. One agent, one hash, one scan — however many turns.

What you see. The agent’s code has not moved, yet Protect opens a new candidate and scans it every morning, or after every conversation. The diff shows one line: a date, a name, a session id.

Why. A scan runs when the snapshot’s hash changes, and the hash is over exactly what was declared. A hand-rolled agent that renders its system prompt per turn — with datetime.now(), the caller’s name, fetched context — and declares that has declared something different every time it runs. Scans are never per conversation; the declared prompt was.

Fix. Declare the template: strip the volatile parts before declare_assets (replace today’s date with a placeholder, leave out per-session context), so an unchanged agent keeps an unchanged hash. See the first rule on the own tool loop page.

Why. AI SDK ≤6 emits spans only when asked.

Fix. experimental_telemetry: { isEnabled: true } on the ToolLoopAgent settings or each generateText / streamText call. v7 needs nothing.

Redline runs are unattended. An agent that asks the user a question has nobody to answer it, so the run ends there. Return the questions as the answer rather than inventing replies.