Talk to us
Shimmy docs

Framework adapters

A framework owns its own LLM client. You never call chat.completions.create yourself, so there is no call site to wrap and no lexical scope to nest — which means the run structure the SDK exists to capture is invisible for exactly the users most likely to want it.

Adapters rebuild the tree from whatever seam each framework provides.

LangChain and LangGraph

The best case. LangChain’s BaseCallbackHandler hands over a runId and a parentRunId — which is already a span tree, just delivered as flat events. So the adapter infers nothing; it reconstructs the tree the framework already knows about.

import { langchainHandler } from '@rfa-labs/shimmy';

const handler = langchainHandler(shimmy, {
  kinds: { classify_intent: 'classification', write_reply: 'generation' },
  minScores: { write_reply: 0.9 },
});

await graph.invoke(input, { callbacks: [handler] });

Because parentRunId is supplied, parallel nodes come out as siblings — the case inference gets actively wrong. A LangGraph fan-out produces one parent with N children, never a chain, and never a phantom cycle when the branches share a step shape.

Tool success and failure are reported automatically. A tool that ran is direct evidence the model’s tool call was well-formed — something the server can only partially verify, because the execution result reaches it a whole request later as opaque text.

Vercel AI SDK

No callback interface, but a middleware seam on any model — which is the better hook anyway, since it sits on the request path.

import { shimmyMiddleware } from '@rfa-labs/shimmy';
import { wrapLanguageModel } from 'ai';

const model = wrapLanguageModel({
  model: openai('gpt-4o'),
  middleware: shimmyMiddleware(shimmy, {
    stepId: 'summarize',
    kind: 'summarization',
  }),
});

Unlike LangChain, the AI SDK’s calls are ordinary function calls in your code, so ambient context already works. The middleware’s job is narrower: carry whatever scope is current onto the request, and give a call a step identity when you have not opened one.

Inside an opt.step() scope, your declaration wins — the middleware default does not override it.

LlamaIndex

Its event bus is flat: no parent pointer. So nesting is reconstructed from event order on a stack, which is correct for sequential pipelines and ambiguous under concurrency.

import { llamaIndexHandler } from '@rfa-labs/shimmy';

const h = llamaIndexHandler(shimmy, { kinds: { synthesize: 'generation' } });
Settings.callbackManager.on('llm-start', h.onEvent);
Settings.callbackManager.on('llm-end', h.onEvent);

Draining on abort

A framework that exits without emitting its end events leaves spans open, and a leaked span makes the next unrelated call look like a child of dead work. Both adapters expose a drain:

handler.closeAll();

Call it on an abort or a process signal.

Rust

There is no Rust adapter, and that is deliberate rather than an omission. No Rust agent framework exposes a callback surface comparable to LangChain’s, so an adapter would be surface with no users. Instrument directly with run() and step() — see instrument an agent.

Next