Models plane

The models plane is the OpenAI-compatible surface: POST /v1/chat/completions, POST /v1/embeddings, GET /v1/models. Base URL https://api.plungeai.com. Auth is your ozk_… account key in Authorization: Bearer (only: X-API-Key is a 401 on this plane, except on /v1/messages, 3.16), billed to your account's wallet — see Authentication & keys. api.plungeai.com/v1/* is a transparent proxy (orchestration/api-gateway/routes/models-proxy.ts) to the inference gateway (inference/gateway/src/); the proxy adds no auth of its own — the inference gateway owns the key, wallet, and metering.

Examples in this chapter that carry # verify are run against the deployed API with an ozk_ key by verify-examples.mjs; the others are verified against the gateway source (inference/gateway/src).

3.1 Endpoints

3.2 Request fields

The support words below are: supported (mapped and sent), partial (only part works — the cell says which), ignored (accepted, no effect), rejected (errors), Ocean-only (a field OpenAI does not define).

Every request routes to exactly one provider shape. openai-shape providers: openai, groq, mistral, deepseek, xai, together, fireworks, cerebras, perplexity, openrouter, moonshot (routing/model-router.ts PROVIDERS). For openai-shape providers the gateway forwards your body after one field split (splitRequestFields, routing/params.ts): Ocean directives (models, sort, compress) and OpenRouter-only fields (provider, route, transforms, plugins, usage, reasoning, include_reasoning, debug, session_id) are removed before the provider sees them (reasoning.effort is sent as reasoning_effort). The sampling knobs top_k, min_p, top_a and repetition_penalty are forwarded to the OpenAI-compatible hosts only: OpenAI itself never receives them, nor any other field it does not define, while the other openai-shape providers keep provider-specific extras and top_k. Those names are returned in the x-ocean-dropped-params response header (3.6) — except min_p, top_a and repetition_penalty, which OpenRouter also defines and which are removed without a mention, like the other OpenRouter-only fields. For anthropic, google and cohere the gateway builds a fresh provider body from the same split; a field the shape cannot take is dropped (ignored). Those names are returned in the x-ocean-dropped-params response header (3.6) — OpenRouter-only fields (min_p, top_a, repetition_penalty among them) are again removed without a mention.

Core fields

Messages & content parts

Tools & structured output

A forced tool call on the Anthropic shape comes back as tool_calls with finish_reason:"tool_calls" and arguments {"city":"Berlin"} (live-verified 2026-10-04). The call needs about 33 completion tokens, above the 16-token cap of # verify fences, so it carries no marker:

curl -s "$API_BASE/v1/chat/completions" \
  -H "Authorization: Bearer $PLUNGE_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model":"anthropic/claude-haiku-4-5","max_tokens":64,"messages":[{"role":"user","content":"Weather in Berlin?"}],"tools":[{"type":"function","function":{"name":"get_weather","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}],"tool_choice":{"type":"function","function":{"name":"get_weather"}}}'

A json_schema response on the Google shape. Gemini 2.5 counts its thinking tokens against max_tokens, so the call needs 512 and carries no # verify marker; message.content is the JSON string {"a":1}:

curl -s "$API_BASE/v1/chat/completions" \
  -H "Authorization: Bearer $PLUNGE_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model":"google/gemini-2.5-flash","max_tokens":512,"messages":[{"role":"user","content":"Return a=1"}],"response_format":{"type":"json_schema","json_schema":{"name":"o","schema":{"type":"object","properties":{"a":{"type":"integer"}},"required":["a"]}}}}'

Sampling & OpenRouter extensions

3.3 Routing: models, sort, failover

routing/select.ts and the runAttempts loop in routes/chat-completions.ts.

• models[] — an ordered candidate list. Each entry is normalized (alias table + provider/model shape); an unresolvable entry is dropped (recorded as reason:"unknown_model"), not fatal, as long as one survives. If every entry is unknown → 404 model_not_found.
• sort — reorders candidates: price (cheapest input+output rate first, from the rate card), latency (lowest rolling p50, unknown providers last), throughput (highest rolling tok/s first). No sort or <2 candidates → input order preserved.
• Attempt loop — one candidate at a time: up to 2 attempts per candidate (one retry on 429/5xx/network), then failover to the next. A non-429 4xx is a caller error on that candidate — no retry, and the loop moves to the next candidate (a retired model id or a stale key on one provider does not sink the request); when no candidate succeeds, the first such error's status and body are returned, so a single-model request gets its own 4xx unchanged. Exhausting every candidate → 502 "All routing candidates failed".
• Outage exclusion — a candidate that exhausts its retries against a real upstream failure is marked out for 30s (KV outage:<provider>); subsequent requests skip it (reason:"outage_excluded") until the window elapses.
• Retry backoff — honors a small upstream Retry-After (capped at 2s), else a flat 250ms.
• Honest model — the response's model is the slug that actually served the request (matters after failover), not the one you asked for, for anthropic/google/cohere and for the non-stream openai path. On the openai-shape stream the per-chunk model is left as the upstream reports it (not rewritten). The full ordered trail is persisted to inference_requests.route_attempts.

3.4 Presets

Set model to @preset/<slug> (policy/presets.ts). The gateway resolves a stored preset (org-specific row wins over a platform-global one) and merges it before candidate resolution. Precedence: your explicit request fields always win — preset params_json fills defaults, preset routing_json.models/sort apply only when you did not send models/sort; the decision log records @preset/<slug> as the requested model. Unknown or inactive slug → 404 preset_not_found. Presets are provisioned by an operator (signed /admin/presets); there is no public create route.

3.5 Response object

Non-stream body is OpenAI chat.completion; stream chunks are chat.completion.chunk (4).

finish_reason per shape (mapAnthropicStop / mapGoogleStop, streaming/sse-translate.ts):

Usage & cache tokens — Anthropic's input_tokens (which excludes cache) is normalized to OpenAI semantics: prompt_tokens includes the cached portion, and prompt_tokens_details breaks it out. Cached tokens bill at cache rates and are subtracted from full-rate input, never double-counted (metering/pricing.ts computeCost).

3.6 Response headers

The money-plane error body carries request_id inside the JSON envelope; note it does not set X-Error-Code (that header is execution-plane only — see Errors, limits & headers).

3.7 Providers & slugs

Address a model as provider/model (e.g. anthropic/claude-sonnet-4-6). Bare aliases in routing/model-router.ts ALIASES auto-prefix: gpt-4o, gpt-4o-mini, gpt-4.1, gpt-4.1-mini, o1, o1-mini, o3-mini, claude-opus-4-7, claude-sonnet-4-6, claude-haiku-4-5, gemini-2.5-pro, gemini-2.5-flash. Any other bare name (no /) → 404 model_not_found.

Discover the live catalog with GET /v1/models (3.8) or the discovery plane GET /v1/discovery/search?kind=models (Discovery). Not every provider slug in the map is priced/active — the catalog is the source of truth.

3.8 GET /v1/models

Requires a key. Returns { "object":"list", "data":[ … ] } built from the rate-card snapshot (routes/models.ts, metering/pricing.ts listCatalog). Each entry:

{
  "id": "anthropic/claude-sonnet-4-6",
  "object": "model",
  "created": 0,
  "owned_by": "anthropic",
  "context_length": 200000,
  "max_output": 8192,
  "pricing": { "in": 3.0, "out": 15.0 },
  "capabilities": { "streaming": true, "tools": true, "vision": true, "embedding": false, "fim": false },
  "name": "Anthropic: claude-sonnet-4-6",
  "canonical_slug": "anthropic/claude-sonnet-4-6",
  "architecture": { "modality": "text+image->text", "input_modalities": ["text","image"], "output_modalities": ["text"], "tokenizer": null, "instruct_type": null },
  "pricing_per_token": { "prompt": "0.000003", "completion": "0.000015", "input_cache_read": "0.0000003", "input_cache_write": "0.00000375" },
  "top_provider": { "context_length": 200000, "max_completion_tokens": 8192, "is_moderated": false },
  "supported_parameters": ["temperature", "top_p", "max_tokens", "max_completion_tokens", "stop", "tools", "tool_choice", "response_format", "top_k"],
  "per_request_limits": null
}

pricing.in/.out are USD per 1M tokens. pricing_per_token is the same price in OpenRouter's format (USD per token, decimal string); supported_parameters lists what the routed provider shape honours (tool parameters only for tool-capable models). vision:false means "not verified", not "verified absent"; max_output:null means unknown; fim is a name heuristic (codestral). This shape differs from OpenRouter's /models — see OpenRouter compatibility.

3.9 Embeddings

POST /v1/embeddings serves openai, mistral, google and cohere models (routes/embeddings.ts, streaming/embeddings-translate.ts); any other provider answers 400 unsupported_provider. Body is standard OpenAI (model, input string or array, optional encoding_format: float or base64); the answer is always OpenAI's {object:"list", data:[{embedding, index}], model, usage}, whatever the provider speaks. When a provider reports no token count (google) the usage is estimated as characters/4. An upstream failure passes through with its own status. BYOK, guardrails, reserve/settle metering, and trial exemption apply exactly as for chat. The price is the rate card's per-token input price; google/gemini-embedding-001 is served (live-verified 2026-10-04) but has no rate-card row yet, so GET /v1/models does not list it and no price is published for it.

3.10 Guardrails

Org- and key-scoped policies (policy/guardrails.ts), merged strictest-wins and enforced before any spend or upstream call. What a developer sees:

Spend caps count inference and connector spend plus outstanding wallet holds. On a models[] request the allow-list/require_byok filters narrow the candidate list per entry; only if none survive do you get the error above.

3.11 Response cache

Exact-match, per-org opt-in, default off (metering/response-cache.ts). Eligibility: stream:false and an explicit temperature:0 (an absent temperature defaults to 1 and is not eligible). The key is a SHA-256 of the entire request body scoped by org, so any differing field (tools, response_format, seed, …) is a different entry. A hit returns the stored body with x-cache: hit, bills $0 (no wallet touch, no upstream call); a miss on an eligible request sets x-cache: miss and stores the response (TTL clamped 60s–24h). A hit can never cross an org boundary.

3.12 BYOK-or-billed

Connect your own provider key (BYOK) in the dashboard; it is stored AES-256-GCM per org (routing/byok.ts, table inference_byok_keys on ocean-billing). When a BYOK key exists for the winning candidate's provider, the gateway uses it and bills only the org's flat byok_routing_fee (cost=0, price=byok_fee). A single-candidate BYOK request with a zero routing fee skips the wallet entirely (no reserve/settle/ledger row); a multi-candidate request always reserves (the winner may not be the zero-fee one). Guardrail require_byok can force BYOK for named providers → 402 byok_required if unmet.

3.13 Billing & metering

From inference/CHARGING-README.md. Every request is metered; only active accounts ever pay — an unknown/missing/stale status resolves to trial (metered, never debited).

• Price = provider cost × org markup (inference_org_settings), computed from the rate-card components (metering/pricing.ts); component-based, per-1M normalized.
• Reserve-then-settle — a conservative max estimate is held before the upstream call and settled at the true price after (metering/billing.ts). A failure releases the hold (no charge). A near-empty wallet → 402 insufficient_balance. An active key over its monthly_limit_usd → 429 insufficient_quota.
• RPM limit applies to everyone (abuse protection, not billing): 429 rate_limit_exceeded at RATE_LIMIT_RPM_DEFAULT (600/min default), atomic per key (Errors, limits & headers).

Posted prices. A model with a posted Ocean price bills at that price whichever host serves it; x-ocean-pricing on every completion says posted, passthrough (cost × markup, above) or byok, and x-ocean-provider names the host. A posted price never exceeds the model maker's own list price; a published change reaches billing within 120 s; an aborted stream settles what streamed at the posted price. The public list: GET /v1/pricing (no key), live at https://plungeai.com/pricing/ (opens in a new tab).

3.14 Not supported today

• No /v1/responses (Responses API), /v1/messages/count_tokens or Message Batches (the Anthropic Messages wire itself is 3.16), /v1/providers, /v1/images, /v1/audio, /v1/moderations or /v1/rerank, and no /v1/embeddings/models list route.
• /v1/embeddings serves openai, mistral, google and cohere models only (3.9); any other provider is 400 unsupported_provider.
• /v1/completions takes prompt as one string; an array is 400 unsupported_parameter.
• usage.completion_tokens_details.reasoning_tokens is passed through only when an openai-shape provider sends it; the anthropic, google and cohere translations do not report it.
• A request that times out is not answered with 408: a network failure fails over to the next candidate and ends as 502 when every candidate failed.
• A service_tier other than auto or default is 400 unsupported_service_tier (live-verified 2026-10-04): the rate card prices the default tier only.
• The OpenRouter-only request fields provider, route, transforms, plugins, include_reasoning, debug and session_id are accepted and ignored: they are removed before the provider sees them, without a mention in x-ocean-dropped-params. Use Ocean models / sort / presets for routing. min_p, top_a and repetition_penalty reach the OpenAI-compatible hosts only (never OpenAI itself, anthropic, google or cohere).
• On anthropic, n, seed, presence_penalty, frequency_penalty, logit_bias, logprobs / top_logprobs, parallel_tool_calls and reasoning_effort are dropped (the Messages API has no such fields) and named in x-ocean-dropped-params; response_format is not enforced when the request also sends tools; images are forwarded without conversion (may 400).
• On google, n, logit_bias, logprobs / top_logprobs, parallel_tool_calls and reasoning_effort are dropped and named in x-ocean-dropped-params, and so are file / input_audio content parts (images are mapped).
• On cohere, the translation is text-only: tools, tool_choice, response_format, top_k, seed, presence_penalty / frequency_penalty and images are dropped without a mention in x-ocean-dropped-params (Cohere tool calling is not built); the fields listed for google are dropped and named.

curl -s "$API_BASE/v1/chat/completions" \
  -H "Authorization: Bearer $PLUNGE_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"max_tokens":8}'

The models plane takes the same ozk_ key as every other plane; a missing or invalid key is 401 invalid_api_key (a legacy sk-ocean- key is also accepted here). GET /v1/models enforces the same:

curl -s "$API_BASE/v1/models" -H "Authorization: Bearer $PLUNGE_API_KEY"

3.15 Account & generation routes

Four read-only routes answer for the calling key's own account. They live on /v1/* (OpenAI shape, key required) and /api/v1/* (OpenRouter shape; /models/{author}/{slug}/endpoints is public there, like OpenRouter's).

A generation row is written after the response completes, so a lookup in the same second can answer 404 — retry after a moment.

curl -s "$API_BASE/api/v1/credits" -H "Authorization: Bearer $PLUNGE_API_KEY"

The gen- id of an /api/v1 completion resolves through GET /v1/generation (live-verified 2026-10-04):

RID=$(curl -s "$API_BASE/api/v1/chat/completions" \
  -H "Authorization: Bearer $PLUNGE_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model":"openai/gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"max_tokens":4}' \
  | python3 -c 'import sys,json;print(json.load(sys.stdin)["id"])')
sleep 3
curl -s "$API_BASE/v1/generation?id=$RID" \
  -H "Authorization: Bearer $PLUNGE_API_KEY"

3.16 Anthropic Messages (POST /v1/messages)

The Anthropic Messages wire on the same money path as /v1/chat/completions (routing, models/sort failover, guardrails, reserve and settle, the x-ocean-* headers of 3.6). An Anthropic SDK or Claude Code points its base URL at the gateway and keeps its own request and response shapes. Handler routes/messages.ts; translation ingress/anthropic-in.ts / ingress/anthropic-out.ts. Also mounted at /api/v1/messages (OpenRouter's Anthropic skin, fallbacks up to 3).

• Auth: x-api-key: ozk_… (with any anthropic-version) or Authorization: Bearer ozk_….
• Request: model (any slug, not only anthropic/*), messages (string or blocks: text, image, tool_use, tool_result, thinking, …), system (string or blocks), max_tokens, stop_sequences, temperature, top_p, top_k, tools, tool_choice, metadata.user_id, thinking, stream; plus Ocean's models, sort, max_cost.
• anthropic/* models: your body is forwarded as sent, with only model (the upstream id) and max_tokens (the capped value, 3.2) set by the gateway — cache_control, thinking and server tools reach Anthropic unchanged.
• Any other model: translated to the chat-completions shape — system → a system message, tool_use → tool_calls, tool_result → tool messages, tools[].input_schema → function.parameters, tool_choice any → required, stop_sequences → stop, metadata.user_id → user. Blocks without a neutral form (thinking, server tools) are not sent.
• Response: an Anthropic message (content text/tool_use blocks, stop_reason, usage with input_tokens excluding cache reads/writes). With stream:true the SSE sequence is message_start, content_block_start / content_block_delta (text_delta, input_json_delta) / content_block_stop, message_delta (stop reason + usage), message_stop; a failure after the first byte ends in an error event.
• Errors: {"type":"error","error":{"type":"<anthropic type>","message":"…"},"request_id":"…"} with the HTTP status of Errors — 401 authentication_error, 402 billing_error, 429 rate_limit_error, 5xx api_error, 504 timeout_error.

curl -s https://gateway.plungeai.com/v1/messages \
  -H "x-api-key: $PLUNGE_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model":"openai/gpt-4o-mini","max_tokens":64,
       "system":"Answer in one sentence.",
       "messages":[{"role":"user","content":"What is a Durable Object?"}]}'

Not yet live-verified (not deployed at time of writing; drill tests/live/inference/anthropic-ingress.sh). api.plungeai.com/v1/messages answers once the One API models proxy forwards the route.