reference/provider-endpoint-notes.md

Provider endpoint notes — operational learnings for calling LLM APIs

Durable, dated record of vendor-endpoint behavior discovered while running experiments in this repo: schema-validation gates, tool_choice policies, reasoning/thinking toggles, and per-model quirks. This is the operational knowledge a new experiment (or agent) needs before wiring an arm against one of these endpoints — distinct from models.json (identity), inference-provider-capabilities.jsonl (HF catalog capability booleans), and the per-topic dataset cards (findings about models). When a fact here came from a probe, the provenance column names the artifact; most live under mcp/tool-schemas/data/.

Convention: add facts with an observed date and a provenance pointer (probe script, CSV, or dataset-card appendix). Facts about endpoints go stale — treat anything older than a few months as "verify before relying on it," and update the entry (with a new date) rather than appending contradictions.

Moonshot — https://api.moonshot.ai/v1 (key MOONSHOT_API_KEY)

• First-party OpenAI-shaped chat completions — this is the vendor's own endpoint design, not a translation gateway, so tools[].function.parameters is the native tool-schema field. (2026-07-18/19; mcp/tool-schemas/companion-space/direct_providers.py _call_moonshot)
• Request-time schema validator ("moonshot flavored json schema") rejects items: false, so a JSON-Schema 2020-12 tuple (prefixItems + items: false) is refused with HTTP 400 before any model sees it: "tools.function.parameters is not a valid moonshot flavored json schema, details: <At path '…items': items must be an object>". Confirmed endpoint-level, not model-level: kimi-k3 and kimi-k2.6 (which honors the same schema 20/20 on deepinfra) get byte-identical rejections. Confirmed a second time on the same model once the HF router began serving moonshotai/Kimi-K3 — K3 via together honors the identical tuple schema 20/20, so the gate is purely Moonshot's request-time validator. (2026-07-19, re-confirmed 2026-07-27; moonshot_family_probe.csv, runs.jsonl kimi-k3 vs kimi-k3-hf, dataset-card appendix)
• Thinking policies differ per hosted model (2026-07-19; probe + platform.kimi.ai docs):
  • kimi-k3: thinking always-on; no thinking object — only top-level reasoning_effort, and only "max" exists.
  • kimi-k2.6: thinking ON by default, but accepts extra_body={"thinking": {"type": "disabled"}} — the only hosted Kimi with an off switch.
  • kimi-k2.7-code: thinking.type accepts only "enabled" — mandatory.
• tool_choice under thinking: a specified tool_choice ({"type":"function","function":{"name":…}}) is refused with 400 "tool_choice 'specified' is incompatible with thinking enabled". On kimi-k3, tool_choice="required" works (with one tool offered, that's equivalent forcing). On hosted kimi-k2.6 with thinking on, even "required" is refused — disable thinking first, after which the specified form works normally. (2026-07-19; _MOONSHOT_TOOLCHOICE_REQUIRED self-adapting fallback in direct_providers.py, probe_moonshot_family.py docstring)
• Gate ordering: the tool_choice gate fires before the schema gate — the reverse of OpenAI, where the schema 400 wins. Means: a surfaced schema rejection under the self-adapting fallback was necessarily produced in "required" mode. (2026-07-19; one-call probe recorded in TODO.md history)
• Zero-balance accounts get HTTP 429 "account … is suspended due to insufficient balance" — a billing state, not a rate limit. (2026-07-18)
• Per vendor docs (2026-07-18, unverified by probe): 1M context, automatic context caching, json_schema + strict: true response_format, partial mode, separate reasoning_content in responses (rides along in our raw dumps).

OpenAI — api.openai.com (key OPENAI_API_KEY)

• /v1/chat/completions refuses function tools at any reasoning effort except "none" (HTTP 400, param: "reasoning_effort"). The gate predates GPT-5.6 (gpt-5.5 returns the byte-identical 400 at explicit reasoning_effort="low"); 5.6 just defaults reasoning on, so it hits the gate on a plain request. The schema gate fires before the reasoning gate. (2026-07-18; gpt56_reasoning_probe.csv)
• /v1/responses is the recommended surface (the reasoning-gate 400 points there; OpenAI's MCP integration runs on it): accepts function tools at default reasoning-on effort, and accepts + honors all six 2020-12 keywords verbatim — including the top-level oneOf that chat.completions deterministically rejects. Tool param shape is flat ({type, name, description, parameters, strict}), tool_choice flat too. Under a forced tool call, models usually skip reasoning (most rows report 0 reasoning tokens). (2026-07-18; direct-responses arm, dataset card)
• strict mode (function-level): rejects report only the first failure, and the envelope requirements (additionalProperties: false + every property required) mask the keyword question. With the envelope added: $ref/$defs accepted; oneOf (anywhere) and if/then genuinely banned; prefixItems unenforceable but silently tolerated; enumNames passes through. Identical validator behind both endpoints. (2026-07-18; strict_probe.csv)

Anthropic — /v1/messages (key ANTHROPIC_API_KEY)

• Top-level combinator gate: oneOf/allOf/anyOf at the schema root is refused ("input_schema does not support oneOf, allOf, or anyOf at the top level") on every tier and generation, with no endpoint escape — single API surface; the only workaround is nesting the union one level down. (2026-07-05..18; grid + anthropic_strict_probe.csv)
• strict tool use (tool-level strict: true, GA): same first-failure envelope masking as OpenAI; accepts $ref/$defs with the envelope; bans oneOf (anywhere), if/then, and — beyond OpenAI — prefixItems (loudly), minimum, maxItems, minItems other than 0/1, and a sibling type next to anyOf. pattern/enumNames/bare consts pass. (2026-07-18; anthropic_strict_probe.csv)

Google — google-genai SDK (keys GOOGLE_API_KEY → GEMINI_API_KEY)

• Two tool-schema fields with very different power: FunctionDeclaration.parameters_json_schema is the raw 2020-12 passthrough (use this); parameters routes through the restricted pydantic Schema (OpenAPI-3.0 subset) that cannot express const/oneOf/if-then/tuple items and rejects them at build time. (2026-07-05; probe_gemini_field.py)
• The transport hides oneOf/anyOf branch consts from the model on every Gemini generation — models see the titles only, so titled-select wire values are reconstructed (3.5-flash) or substituted with the title (3.1 line). (2026-07-13; title_const_probe.csv)
• SDK client: timeout is in milliseconds; bound 429 backoff via HttpRetryOptions or a rate-limited call stalls the pool. (2026-07-05; direct_providers.py _google_client)

Hugging Face Inference Providers router — https://router.huggingface.co/v1 (one HF_TOKEN)

• OpenAI-compatible gateway; model ids are HF Hub ids; pin a serving provider via the <hub-id>:<provider> model-id suffix, and record the router's echoed model id per row — a verdict on an open model is a (model × serving host) fact, hosts flip verdicts (see the cross-host and grammar appendices: fireworks-ai rejects schemas deepinfra serves; deepinfra and together are free decoders, not grammar-masking). (2026-07-05..19; cross_host.csv, grammar_probe.csv, moonshot_family_probe.csv)
• Provider capabilities drift week to week — same model gains/loses structured output by provider (e.g. DeepSeek-V4-Flash was tools-only everywhere on 2026-07-05, had a structured-output route by 2026-07-18). Re-pull inference-provider-capabilities.jsonl before relying on it (python reference/refresh-capabilities.py --diff --summary). (2026-07-18)
• The catalog also gains and loses whole routes, fast. Between the 2026-07-18 and 2026-07-27 pulls: 10 (model, provider) pairs appeared, 7 disappeared, 24 flipped a capability boolean — and two additions landed between two pulls minutes apart. Concretely for this repo's roster: moonshotai/Kimi-K3 arrived (together, tools + structured); Qwen/Qwen3.6-35B-A3B lost deepinfra entirely (only scaleway still serves it with tools) — and deepinfra was its pinned host; thinkingmachines/Inkling flipped supports_tools false on together (its pinned host) while gaining a deepinfra route; XiaomiMiMo/MiMo-V2.5-Pro flipped to tools + structured on deepinfra, having been gate-excluded as unservable. A pin recorded from an older snapshot may name a route that no longer exists — re-verify before re-running a cell. (2026-07-27; reference/refresh-capabilities.py --diff)
• A model can be API-only one week and on the router the next: Kimi K3 had no Hub weights and no router route on 2026-07-18, and by 2026-07-27 was serving on together. Worth re-checking before asserting "vendor API is the only route" for any recent model. (2026-07-27)
• probe_hf.py merges a scoped run into hf_feasibility.json: HF_PROBE_MODELS=x re-probes only x and preserves every other label's pin (it prints (scoped run: merged prior manifest entries for the N unprobed labels)). This was a real footgun before — a scoped run used to write the manifest wholesale and silently wipe the other pins — so on an older checkout, run the full probe instead. Either way, diff the pins against the committed manifest afterwards. (fixed by 2026-07-27; verified adding kimi-k3-hf)