reference/provider-endpoint-notes.md

# Provider endpoint notes — operational learnings for calling LLM APIs

Durable, dated record of vendor-endpoint **behavior** discovered while running experiments in this repo: schema-validation gates, tool_choice policies, reasoning/thinking toggles, and per-model quirks. This is the *operational* knowledge a new experiment (or agent) needs before wiring an arm against one of these endpoints — distinct from `models.json` (identity), `inference-provider-capabilities.jsonl` (HF catalog capability booleans), and the per-topic dataset cards (findings about *models*). When a fact here came from a probe, the provenance column names the artifact; most live under `mcp/tool-schemas/data/`.

Convention: add facts with an observed date and a provenance pointer (probe script, CSV, or dataset-card appendix). Facts about endpoints go stale — treat anything older than a few months as "verify before relying on it," and update the entry (with a new date) rather than appending contradictions.

## Moonshot — `https://api.moonshot.ai/v1` (key `MOONSHOT_API_KEY`)

- **First-party OpenAI-shaped chat completions** — this is the vendor's own endpoint design, not a translation gateway, so `tools[].function.parameters` is the native tool-schema field. (2026-07-18/19; `mcp/tool-schemas/companion-space/direct_providers.py` `_call_moonshot`)
- **Request-time schema validator ("moonshot flavored json schema") rejects `items: false`**, so a JSON-Schema 2020-12 tuple (`prefixItems` + `items: false`) is refused with HTTP 400 before any model sees it: *"tools.function.parameters is not a valid moonshot flavored json schema, details: \<At path '…items': items must be an object\>"*. Confirmed endpoint-level, not model-level: kimi-k3 and kimi-k2.6 (which honors the same schema 20/20 on deepinfra) get byte-identical rejections. **Confirmed a second time on the same model** once the HF router began serving `moonshotai/Kimi-K3` — K3 via `together` honors the identical tuple schema 20/20, so the gate is purely Moonshot's request-time validator. (2026-07-19, re-confirmed 2026-07-27; `moonshot_family_probe.csv`, `runs.jsonl` `kimi-k3` vs `kimi-k3-hf`, dataset-card appendix)
- **Thinking policies differ per hosted model** (2026-07-19; probe + platform.kimi.ai docs):
  - `kimi-k3`: thinking **always-on**; no `thinking` object — only top-level `reasoning_effort`, and only `"max"` exists.
  - `kimi-k2.6`: thinking ON by default, but accepts `extra_body={"thinking": {"type": "disabled"}}` — the only hosted Kimi with an off switch.
  - `kimi-k2.7-code`: `thinking.type` accepts only `"enabled"` — mandatory.
- **tool_choice under thinking**: a *specified* tool_choice (`{"type":"function","function":{"name":…}}`) is refused with 400 *"tool_choice 'specified' is incompatible with thinking enabled"*. On kimi-k3, `tool_choice="required"` works (with one tool offered, that's equivalent forcing). On hosted kimi-k2.6 with thinking on, even `"required"` is refused — disable thinking first, after which the specified form works normally. (2026-07-19; `_MOONSHOT_TOOLCHOICE_REQUIRED` self-adapting fallback in `direct_providers.py`, `probe_moonshot_family.py` docstring)
- **Gate ordering: the tool_choice gate fires *before* the schema gate** — the reverse of OpenAI, where the schema 400 wins. Means: a surfaced schema rejection under the self-adapting fallback was necessarily produced in `"required"` mode. (2026-07-19; one-call probe recorded in TODO.md history)
- Zero-balance accounts get HTTP 429 *"account … is suspended due to insufficient balance"* — a billing state, not a rate limit. (2026-07-18)
- Per vendor docs (2026-07-18, unverified by probe): 1M context, automatic context caching, `json_schema` + `strict: true` response_format, partial mode, separate `reasoning_content` in responses (rides along in our `raw` dumps).

## OpenAI — `api.openai.com` (key `OPENAI_API_KEY`)

- **`/v1/chat/completions` refuses function tools at any reasoning effort except `"none"`** (HTTP 400, `param: "reasoning_effort"`). The gate predates GPT-5.6 (gpt-5.5 returns the byte-identical 400 at explicit `reasoning_effort="low"`); 5.6 just defaults reasoning on, so it hits the gate on a plain request. The **schema gate fires before the reasoning gate**. (2026-07-18; `gpt56_reasoning_probe.csv`)
- **`/v1/responses` is the recommended surface** (the reasoning-gate 400 points there; OpenAI's MCP integration runs on it): accepts function tools at default reasoning-on effort, and accepts + honors all six 2020-12 keywords verbatim — including the top-level `oneOf` that chat.completions deterministically rejects. Tool param shape is **flat** (`{type, name, description, parameters, strict}`), tool_choice flat too. Under a forced tool call, models usually skip reasoning (most rows report 0 reasoning tokens). (2026-07-18; `direct-responses` arm, dataset card)
- **strict mode** (function-level): rejects report only the *first* failure, and the envelope requirements (`additionalProperties: false` + every property `required`) mask the keyword question. With the envelope added: `$ref`/`$defs` accepted; `oneOf` (anywhere) and `if`/`then` genuinely banned; `prefixItems` unenforceable but silently tolerated; `enumNames` passes through. Identical validator behind both endpoints. (2026-07-18; `strict_probe.csv`)

## Anthropic — `/v1/messages` (key `ANTHROPIC_API_KEY`)

- **Top-level combinator gate**: `oneOf`/`allOf`/`anyOf` at the schema root is refused ("input_schema does not support oneOf, allOf, or anyOf at the top level") on every tier and generation, with **no endpoint escape** — single API surface; the only workaround is nesting the union one level down. (2026-07-05..18; grid + `anthropic_strict_probe.csv`)
- **strict tool use** (tool-level `strict: true`, GA): same first-failure envelope masking as OpenAI; accepts `$ref`/`$defs` with the envelope; bans `oneOf` (anywhere), `if`/`then`, and — beyond OpenAI — `prefixItems` (loudly), `minimum`, `maxItems`, `minItems` other than 0/1, and a sibling `type` next to `anyOf`. `pattern`/`enumNames`/bare `const`s pass. (2026-07-18; `anthropic_strict_probe.csv`)

## Google — `google-genai` SDK (keys `GOOGLE_API_KEY` → `GEMINI_API_KEY`)

- Two tool-schema fields with very different power: `FunctionDeclaration.parameters_json_schema` is the **raw 2020-12 passthrough** (use this); `parameters` routes through the restricted pydantic `Schema` (OpenAPI-3.0 subset) that cannot express `const`/`oneOf`/`if-then`/tuple items and rejects them at build time. (2026-07-05; `probe_gemini_field.py`)
- **The transport hides `oneOf`/`anyOf` branch `const`s from the model** on every Gemini generation — models see the `title`s only, so titled-select wire values are reconstructed (3.5-flash) or substituted with the title (3.1 line). (2026-07-13; `title_const_probe.csv`)
- SDK client: `timeout` is in **milliseconds**; bound 429 backoff via `HttpRetryOptions` or a rate-limited call stalls the pool. (2026-07-05; `direct_providers.py` `_google_client`)

## Hugging Face Inference Providers router — `https://router.huggingface.co/v1` (one `HF_TOKEN`)

- OpenAI-compatible gateway; model ids are HF Hub ids; pin a serving provider via the `<hub-id>:<provider>` model-id suffix, and record the router's echoed model id per row — **a verdict on an open model is a `(model × serving host)` fact**, hosts flip verdicts (see the cross-host and grammar appendices: fireworks-ai rejects schemas deepinfra serves; deepinfra and together are free decoders, not grammar-masking). (2026-07-05..19; `cross_host.csv`, `grammar_probe.csv`, `moonshot_family_probe.csv`)
- Provider capabilities drift week to week — same model gains/loses structured output by provider (e.g. DeepSeek-V4-Flash was tools-only everywhere on 2026-07-05, had a structured-output route by 2026-07-18). Re-pull `inference-provider-capabilities.jsonl` before relying on it (`python reference/refresh-capabilities.py --diff --summary`). (2026-07-18)
- **The catalog also gains and loses whole routes, fast.** Between the 2026-07-18 and 2026-07-27 pulls: 10 `(model, provider)` pairs appeared, 7 disappeared, 24 flipped a capability boolean — and two additions landed *between two pulls minutes apart*. Concretely for this repo's roster: `moonshotai/Kimi-K3` **arrived** (together, tools + structured); `Qwen/Qwen3.6-35B-A3B` **lost deepinfra entirely** (only scaleway still serves it with tools) — and deepinfra was its pinned host; `thinkingmachines/Inkling` flipped `supports_tools` **false** on together (its pinned host) while gaining a deepinfra route; `XiaomiMiMo/MiMo-V2.5-Pro` flipped to tools + structured on deepinfra, having been gate-excluded as unservable. **A pin recorded from an older snapshot may name a route that no longer exists — re-verify before re-running a cell.** (2026-07-27; `reference/refresh-capabilities.py --diff`)
- A model can be **API-only one week and on the router the next**: Kimi K3 had no Hub weights and no router route on 2026-07-18, and by 2026-07-27 was serving on `together`. Worth re-checking before asserting "vendor API is the only route" for any recent model. (2026-07-27)
- `probe_hf.py` **merges** a scoped run into `hf_feasibility.json`: `HF_PROBE_MODELS=x` re-probes only `x` and preserves every other label's pin (it prints `(scoped run: merged prior manifest entries for the N unprobed labels)`). This was a real footgun before — a scoped run used to write the manifest wholesale and silently wipe the other pins — so on an older checkout, run the full probe instead. Either way, diff the pins against the committed manifest afterwards. (fixed by 2026-07-27; verified adding `kimi-k3-hf`)