One API Surface Over Every Model Dialect

Providers diverge in four places, and it is worth being precise about them because the whole design follows from the list. The message shape: where the system turn goes, whether content is a string or typed blocks, whether consecutive turns from one role are legal. The parameter names: the same idea called different things, sometimes a scalar here and an array there. The stream protocol: both server-sent events, different event names, different delta semantics. And the error body: the same failure reported at a different path in a different envelope.

Each difference is trivial in isolation. The problem is that there are four of them per provider, and the combination is what a client has to learn.

A provider is a declarative object. It names the auth headers it needs, with a placeholder where the secret goes. It names a message style and a stream style, by string, rather than carrying the code for either. It maps each OpenAI parameter to a provider field with a small rule — rename it, wrap a scalar into an array, fill it from the catalog row when the provider requires it and the client omitted it, or hand the value to a named translator when the shape has to change rather than the name. And it names the dot paths where the provider's error message and type live.

The engine reads that and does the work. The whole Anthropic spec is nineteen lines and contains no logic at all.

The honest boundary: this is only configuration for providers whose behaviour is already expressible in existing styles. A genuinely new protocol needs a new style, which is real code in the engine. The claim is that the second provider on a known protocol is nearly free, not that any provider is.

The temptation with a gateway is to write two response paths — one that forwards a stream and one that returns a whole body — and then spend a year fixing the places where they disagree. Onefold has one.

Provider bytes become server-sent event frames, frames become dialect-neutral events, and there are only six kinds of those: some text, a tool call opened, a fragment of that call's arguments, a usage report, a finish reason, an error. Both response modes consume that same sequence. Streaming re-emits each event as an OpenAI chunk; non-streaming consumes the sequence to the end and assembles one body from it.

Non-streaming is therefore not a second implementation. It is the streaming path, read to completion before anything is written. The two can not disagree about what the model said, because they are reading the same events.

Tool calls are where dialects stop being cosmetic. Anthropic indexes every content block in a response against one counter — text blocks and tool blocks share it. OpenAI numbers tool calls densely from zero and does not count text at all. A response that goes text, tool, text, tool produces Anthropic blocks one and three, which the client expects to see as tool calls zero and one.

So the translator carries a map from block index to tool index, built as the response arrives. That map is meaningless outside a single response, which is why a stream style is a factory rather than a function: the engine constructs a fresh translator per request, and any state a dialect needs lives in the closure rather than in a module.

The second half is that tool arguments stream as partial JSON. Not an object, not valid JSON on its own — a fragment. The gateway forwards fragments verbatim and lets the client concatenate them, because the alternative is buffering until the object closes, and a gateway that waits for a whole tool call before saying anything has quietly stopped streaming.

Every request that reaches a provider produces one record: which model, which provider, status, finish reason, token counts both ways, latency, whether it streamed, which key and organisation, and what it cost. That record is written after the response has been handed to the caller, on the runtime's post-response hook, and a failure to write it is a log line rather than an error.

It lands in a Postgres staging table first and is drained into ClickHouse once a minute. The hop exists for an unglamorous reason — the Worker runtime cannot reach ClickHouse directly — but it bought something real. The analytics store is append-only, queried by aggregation over time ranges, and has no business sharing a transactional database with the catalog. The staging table makes the boundary explicit and gives the drain somewhere to retry from.

Retries are safe because the destination table collapses rows with the same id on merge, so a batch replayed after a partial failure produces one row rather than two. Idempotency is a property of the schema instead of a discipline the drain script has to maintain.