Cookbook: adding an LLM adapter
How to connect a new model provider. Reference implementations: packages/llm/llm-deepseek (direct HTTP, SSE framed by eventsource-parser) and packages/llm/llm-pi-ai (wrapping an LLM library). Read the StreamChunk doc in packages/llm/llm/src/types.ts first — it records the protocol conventions both adapters were verified against.
The shape
class MyAdapter extends LlmAdapter {
async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> { … }
}
export const name = 'llm-myprovider'
export const inject = ['llm']
export const Config: z<Config> = z.object({ apiKey: z.string(), … })
export function apply(ctx: Context, config: Config) {
ctx.llm.registerAdapter(['my-provider'], new MyAdapter(…))
}
Registration is effect-based (HMR-safe); one adapter per provider route — duplicates throw, and multi-route registration is all-or-nothing. options.provider selects the adapter and options.model is the provider model id, so a dynamic catalog adapter can serve new models without lifecycle reconfiguration. Secrets are cordis-native: schemastery Config with env fallbacks, fed from cordis.yml via !!js process.env.MY_KEY. Never read ad-hoc key files in code.
Protocol obligations (the contract two implementations verified)
- Emit
usageBEFOREfinish; emit NOTHING afterfinish. The robust way: buffer finish/usage until the provider's end-of-stream marker, then flush (handles providers that send trailing usage-only chunks). - Tool-call
argumentsare RAW JSON strings end-to-end; stream fragments asargumentsDelta. If your provider hands back parsed objects, re-stringify atblock-end. - Allocate block
indexes in first-seen stream order; reuse the index for every delta of the same block. - Errors have exactly two sanctioned paths: THROW from
stream()(transport and protocol failures — useLlmErrorwith a stable code), or end the stream withfinish {kind: 'error' | 'aborted'}(provider in-band failures). Consumers handle both; pick per failure class and document it. - Honor
options.signal(pass it to fetch / your SDK). - A
GenerateOptionsfield your provider cannot honor (e.g. astoplist on a provider without stop sequences): throwLlmError(..., 'UNSUPPORTED')rather than silently dropping it. - If the provider requires response ids, signatures, or other native metadata on follow-up calls, emit the minimal lossless-JSON projection as
finish.replayState. Validate it when rebuilding history.LlmRuntimepasses it only when the historical provider route and target provider route are currently owned by the exact same adapter instance; your adapter decides whether same-model, cross-model, or cross-provider restoration is legal. Never infer native replay from provider/model names alone when state is absent.
Provider-specific thinking-mode toggles remain in the adapter's Config. Exact model metadata uses one provider-neutral capability seam: implement resolveModel() with provider/model identity and optional context and reasoning fields, declare a configured defaultEffort only when one exists, and honor the resolver's optional AbortSignal. Reasoning efforts are ordered opaque ids mapped to provider requests by the adapter. Preserve the adapter's authoritative selectable list, including an adapter-defined off when supported, without exposing final wire spellings or clamping unsupported values; an id need not equal its wire representation.
Implementation structure
Keep wire types, request serialization, transport parsing, chunk translation, and the adapter class as separate responsibilities; llm-deepseek is the reference layout.
Verification
Follow the repository testing policy, which owns adapter coverage, real-provider checks, and published-entry requirements.