Make reasoning policy provider-neutral and client-aware (#1148)
Browse files## Problem
FCC reduced reasoning to global and route booleans, mixing client
intent, configuration, provider wire capabilities, output visibility,
and history replay. That discarded named client efforts, encouraged
model-name checks, and made provider behavior inconsistent.
## Changes
| Before | After |
| --- | --- |
| Admin exposed global and route thinking toggles. | Admin exposes
**Off**, **From client**, **Low**, **Medium**, **High**, **X-High**, and
**Max**; Fable, Opus, Sonnet, and Haiku also expose **Inherit**. |
| Request intent was repeatedly reduced to a boolean across routing and
providers. | The application boundary resolves one immutable
`ReasoningPolicy` with independent control, named effort, and exact
positive token budget. |
| Provider adapters could infer reasoning behavior from upstream model
names or versions. | Provider profiles translate only documented
provider-wide wire capabilities; architecture and contributor rules
prohibit model-specific reasoning branches. |
| Gateway reasoning controls were ad hoc. |
[OpenRouter](https://openrouter.ai/docs/guides/best-practices/reasoning-tokens)
and [Vercel AI
Gateway](https://vercel.com/docs/ai-gateway/models-and-providers) use
documented reasoning objects, including exact budgets where
representable. |
| Named effort forwarding was inconsistent or absent. |
[Gemini](https://ai.google.dev/gemini-api/docs/openai),
[Ollama](https://docs.ollama.com/api/openai-compatibility), [LM
Studio](https://lmstudio.ai/changelog/lmstudio-v0.4.8),
[Fireworks](https://docs.fireworks.ai/guides/querying-text-models/reasoning),
[Cohere](https://docs.cohere.com/docs/compatibility-api),
[Wafer](https://docs.wafer.ai/serverless/api-reference),
[Groq](https://console.groq.com/docs/reasoning),
[Cerebras](https://inference-docs.cerebras.ai/capabilities/reasoning),
[SambaNova](https://docs.sambanova.ai/docs/api-reference/chat-completions/create-chat-based-completion),
and
[Mistral](https://docs.mistral.ai/studio-api/conversations/reasoning)
receive their documented named vocabularies with explicit provider-owned
downgrades. |
| Boolean thinking controls were mixed into shared conversion. |
[DeepSeek](https://api-docs.deepseek.com/guides/thinking_mode/),
[Kimi](https://platform.kimi.ai/docs/guide/use-kimi-k2-thinking-model),
[Z.ai](https://docs.z.ai/guides/capabilities/thinking-mode), [Cloudflare
Workers
AI](https://developers.cloudflare.com/changelog/post/2026-04-20-kimi-k2-6-workers-ai/),
and [NVIDIA
NIM](https://docs.nvidia.com/nim/large-language-models/1.15.0/thinking-budget-control.html)
use provider-owned thinking-object or chat-template controls. |
| Effort names and output limits could become fabricated reasoning
budgets. | Exact budgets remain exact and are forwarded only through
documented fields for OpenRouter, Fireworks, LM Studio, NIM, and
[llama.cpp](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md);
named efforts and output limits are never converted into token budgets.
|
| New-turn reasoning and prior-turn replay shared one switch. | Every
profile independently declares native reasoning replay, `<think>` tag
replay, provider-specific replay, or no replay; **Off** suppresses new
reasoning output without corrupting required history. |
| Providers without a stable generic compute control received guessed
controls. |
[MiniMax](https://platform.minimax.io/docs/api-reference/text-openai-api)
requests split output only, while [GitHub
Models](https://docs.github.com/en/rest/models/inference), [Hugging Face
Inference
Providers](https://huggingface.co/docs/inference-providers/en/tasks/chat-completion),
Codestral, and OpenCode keep provider defaults and use only their
explicit replay profile. |
| OpenAI Responses effort became a lossy Anthropic thinking boolean. |
Responses preserves `reasoning.effort` through `output_config`, then
resolves it through the same application policy as Messages without
inventing a budget. |
| Legacy booleans remained the persisted contract. | FCC-owned dotenv
files migrate to typed `REASONING_*` values, explicit env files receive
an actionable warning, documentation describes the ownership boundary,
and the package advances to 4.8.0. |
| Reasoning behavior was covered by scattered boolean assertions. | New
policy, routing, encoder, provider, Admin, migration, Responses, and
smoke contracts pass all five local CI checks: 2,368 tests passed, 40
skipped; 92 smoke tests collect and both live config migration checks
pass. |
<!-- greptile_comment -->
<details open><summary><h3>Greptile Summary</h3></summary>
This PR makes reasoning policy client-aware and independent of provider
model names. The main changes are:
- Adds one immutable reasoning policy resolved at the application
boundary.
- Adds typed root and route reasoning settings with Admin UI support.
- Moves wire controls and history replay behavior into provider
profiles.
- Migrates owned dotenv files from legacy thinking booleans.
- Expands provider, routing, migration, API, and smoke coverage.
</details>
<h3>Confidence Score: 5/5</h3>
This looks safe to merge.
No blocking issues found in the changed code.
<details><summary><h3><a href="https://www.greptile.com/trex"><img
alt="T-Rex"
src="https://greptile-static-assets.s3.amazonaws.com/trex/trex_green.svg"
height="20" align="absmiddle"></a> T-Rex Logs</h3></summary>
**What T-Rex did**
- Ran the contract-validation test suite with the specified test
modules, and the tests reported 78 passed in 1.53s with exit code 0.
- Reviewed the complete captured output artifact
reasoning-contract-02-after.log to verify the final test outcomes and
successful contract validation.
<a
href="https://app.greptile.com/trex/runs/14792858/artifacts"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://greptile-static-assets.s3.amazonaws.com/badges/ViewAllArtifactsDark.svg?v=4"><source
media="(prefers-color-scheme: light)"
srcset="https://greptile-static-assets.s3.amazonaws.com/badges/ViewAllArtifacts.svg?v=4"><img
alt="View all artifacts"
src="https://greptile-static-assets.s3.amazonaws.com/badges/ViewAllArtifacts.svg?v=4"></picture></a>
<sub><a href="https://www.greptile.com/trex"><img alt="T-Rex"
src="https://greptile-static-assets.s3.amazonaws.com/trex/trex_green.svg"
height="14" align="absmiddle"></a> Ran code and verified through
T-Rex</sub>
</details>
<details open><summary><h3>Important Files Changed</h3></summary>
| Filename | Overview |
|----------|----------|
| src/free_claude_code/config/env_migrations.py | Migrates legacy
reasoning booleans in owned dotenv files and warns for explicit
environment files. |
| src/free_claude_code/application/reasoning.py | Resolves client
controls and configured preferences into one provider-neutral reasoning
policy. |
| src/free_claude_code/application/routing.py | Carries route-level
reasoning preferences into request-scoped policy resolution. |
| src/free_claude_code/providers/openai_chat/reasoning.py | Provides
shared provider encoders for reasoning controls and replay behavior. |
</details>
<sub>Reviews (2): Last reviewed commit: ["chore: release reasoning
controls as
4.8..."](https://github.com/alishahryar1/free-claude-code/commit/9d4be767f7dbdca5709474012f43dcdc6f4347e3)
| [Re-trigger
Greptile](https://app.greptile.com/api/retrigger?id=44984039)</sub>
<!-- /greptile_comment -->
- .env.example +9 -8
- AGENTS.md +1 -0
- ARCHITECTURE.md +72 -21
- CLAUDE.md +1 -0
- README.md +6 -0
- pyproject.toml +1 -1
- smoke/capabilities.py +6 -5
- smoke/features.py +1 -1
- smoke/product/test_config_extensibility_product_live.py +16 -16
- smoke/product/test_nvidia_nim_cli_product_live.py +1 -1
- smoke/product/test_openrouter_free_cli_product_live.py +1 -1
- smoke/product/test_provider_product_live.py +6 -4
- smoke/product/test_runtime_ownership_product_live.py +23 -1
- src/free_claude_code/api/admin_static/admin.js +5 -14
- src/free_claude_code/api/handlers/messages.py +3 -5
- src/free_claude_code/application/execution.py +9 -3
- src/free_claude_code/application/ports.py +3 -2
- src/free_claude_code/application/reasoning.py +99 -0
- src/free_claude_code/application/routing.py +61 -40
- src/free_claude_code/cli/commands.py +2 -2
- src/free_claude_code/config/admin/manifest.py +77 -37
- src/free_claude_code/config/admin/values.py +15 -2
- src/free_claude_code/config/env_migrations.py +65 -12
- src/free_claude_code/config/reasoning.py +24 -0
- src/free_claude_code/config/settings.py +25 -14
- src/free_claude_code/core/__init__.py +14 -0
- src/free_claude_code/core/anthropic/request_snapshot.py +0 -1
- src/free_claude_code/core/gateway_model_ids.py +4 -5
- src/free_claude_code/core/openai_responses/input.py +3 -3
- src/free_claude_code/core/openai_responses/reasoning.py +5 -5
- src/free_claude_code/core/reasoning.py +93 -0
- src/free_claude_code/providers/base.py +6 -21
- src/free_claude_code/providers/cloudflare/client.py +8 -27
- src/free_claude_code/providers/deepseek/client.py +12 -5
- src/free_claude_code/providers/deepseek/compat.py +26 -12
- src/free_claude_code/providers/gemini/client.py +33 -6
- src/free_claude_code/providers/gemini/quirks.py +5 -15
- src/free_claude_code/providers/github_models/client.py +4 -12
- src/free_claude_code/providers/lmstudio/client.py +31 -27
- src/free_claude_code/providers/mistral/client.py +14 -10
- src/free_claude_code/providers/mistral/reasoning.py +8 -12
- src/free_claude_code/providers/nvidia_nim/client.py +12 -5
- src/free_claude_code/providers/nvidia_nim/request_options.py +36 -15
- src/free_claude_code/providers/open_router/client.py +15 -46
- src/free_claude_code/providers/openai_chat/__init__.py +10 -0
- src/free_claude_code/providers/openai_chat/extra_body.py +27 -0
- src/free_claude_code/providers/openai_chat/profiles.py +179 -93
- src/free_claude_code/providers/openai_chat/provider.py +27 -21
- src/free_claude_code/providers/openai_chat/reasoning.py +146 -0
- src/free_claude_code/providers/openai_chat/request_policy.py +8 -14
|
@@ -139,14 +139,15 @@ FCC_SMOKE_OPENROUTER_FREE_MODELS=
|
|
| 139 |
FCC_SMOKE_OPENROUTER_FREE_EXTRA_MODELS=
|
| 140 |
|
| 141 |
|
| 142 |
-
#
|
| 143 |
-
#
|
| 144 |
-
#
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
|
|
|
|
| 150 |
|
| 151 |
|
| 152 |
# Provider config
|
|
|
|
| 139 |
FCC_SMOKE_OPENROUTER_FREE_EXTRA_MODELS=
|
| 140 |
|
| 141 |
|
| 142 |
+
# Reasoning policy
|
| 143 |
+
# Root: off | client | low | medium | high | xhigh | max
|
| 144 |
+
# Route overrides additionally accept inherit. "client" preserves the CLI's effort;
|
| 145 |
+
# providers translate only controls documented by their API.
|
| 146 |
+
REASONING_POLICY=client
|
| 147 |
+
REASONING_FABLE=inherit
|
| 148 |
+
REASONING_OPUS=inherit
|
| 149 |
+
REASONING_SONNET=inherit
|
| 150 |
+
REASONING_HAIKU=inherit
|
| 151 |
|
| 152 |
|
| 153 |
# Provider config
|
|
@@ -35,6 +35,7 @@
|
|
| 35 |
- **DRY**: Extract shared base classes to eliminate duplication. Prefer composition over copy-paste.
|
| 36 |
- **Encapsulation**: Use accessor methods for internal state (e.g. `set_current_task()`), not direct `_attribute` assignment from outside.
|
| 37 |
- **Provider-specific config**: Keep provider-specific fields (e.g. `nim_settings`) in provider constructors, not in the base `ProviderConfig`.
|
|
|
|
| 38 |
- **Dead code**: Remove unused code, legacy systems, and hardcoded values. Use settings/config instead of literals (e.g. `settings.provider_type` not `"nvidia_nim"`).
|
| 39 |
- **Performance**: Use list accumulation for strings (not `+=` in loops), cache env vars at init, prefer iterative over recursive when stack depth matters.
|
| 40 |
- **Platform-agnostic naming**: Use generic names (e.g. `PLATFORM_EDIT`) not platform-specific ones (e.g. `TELEGRAM_EDIT`) in shared code.
|
|
|
|
| 35 |
- **DRY**: Extract shared base classes to eliminate duplication. Prefer composition over copy-paste.
|
| 36 |
- **Encapsulation**: Use accessor methods for internal state (e.g. `set_current_task()`), not direct `_attribute` assignment from outside.
|
| 37 |
- **Provider-specific config**: Keep provider-specific fields (e.g. `nim_settings`) in provider constructors, not in the base `ProviderConfig`.
|
| 38 |
+
- **Model-independent reasoning**: Resolve client reasoning intent once at the application boundary; provider adapters translate documented provider capabilities. Never branch on upstream model names or versions to choose reasoning behavior.
|
| 39 |
- **Dead code**: Remove unused code, legacy systems, and hardcoded values. Use settings/config instead of literals (e.g. `settings.provider_type` not `"nvidia_nim"`).
|
| 40 |
- **Performance**: Use list accumulation for strings (not `+=` in loops), cache env vars at init, prefer iterative over recursive when stack depth matters.
|
| 41 |
- **Platform-agnostic naming**: Use generic names (e.g. `PLATFORM_EDIT`) not platform-specific ones (e.g. `TELEGRAM_EDIT`) in shared code.
|
|
@@ -312,9 +312,15 @@ Model routing configuration is tiered:
|
|
| 312 |
|
| 313 |
- `MODEL` is the fallback provider-prefixed model ref.
|
| 314 |
- `MODEL_FABLE`, `MODEL_OPUS`, `MODEL_SONNET`, and `MODEL_HAIKU` override Claude model tiers.
|
| 315 |
-
- `
|
| 316 |
-
|
| 317 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 318 |
|
| 319 |
[config/model_refs.py](src/free_claude_code/config/model_refs.py) owns provider-prefixed model ref
|
| 320 |
parsing and configured `MODEL*` inventory. API routing and provider validation
|
|
@@ -434,7 +440,7 @@ sequenceDiagram
|
|
| 434 |
Route->>Manager: acquire current generation
|
| 435 |
Manager-->>Route: Lease(settings, provider resolver)
|
| 436 |
Route->>Handler: create message
|
| 437 |
-
Handler->>Router: resolve model and
|
| 438 |
Handler->>Handler: server tools or optimizations
|
| 439 |
Handler->>Exec: stream routed request
|
| 440 |
Exec->>Lease: resolve provider
|
|
@@ -465,10 +471,15 @@ If the incoming model is not direct, `ModelRouter` maps it by Claude tier. Names
|
|
| 465 |
containing `fable`, `opus`, `sonnet`, or `haiku` use the matching tier override when set,
|
| 466 |
otherwise they fall back to `MODEL`.
|
| 467 |
|
| 468 |
-
The router also
|
| 469 |
-
|
| 470 |
-
|
| 471 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 472 |
|
| 473 |
`GET /v1/models` advertises:
|
| 474 |
|
|
@@ -480,9 +491,11 @@ decision; provider catalog metadata does not cross the application boundary.
|
|
| 480 |
Provider model discovery and optional thinking metadata live in the
|
| 481 |
application-level catalog owned by `ProviderRuntimeManager`.
|
| 482 |
`ProviderModelInfo.supports_thinking` alone owns discovered per-model thinking
|
| 483 |
-
support
|
| 484 |
-
|
| 485 |
-
|
|
|
|
|
|
|
| 486 |
|
| 487 |
Codex-specific model picker shaping stays out of this route. `fcc-codex` fetches
|
| 488 |
the same `/v1/models` response at launch, converts FCC gateway IDs into
|
|
@@ -536,7 +549,7 @@ compatibility layer.
|
|
| 536 |
[providers/base.py](src/free_claude_code/providers/base.py) defines provider-internal construction and lifecycle contracts:
|
| 537 |
|
| 538 |
- `ProviderConfig`: shared provider settings such as API key, base URL, rate
|
| 539 |
-
limits, timeouts, proxy,
|
| 540 |
value whose base URL has already been resolved from the catalog.
|
| 541 |
- `BaseProvider`: the abstract implementation base for cleanup, model listing,
|
| 542 |
explicit preflight, and `stream_response()`.
|
|
@@ -544,7 +557,8 @@ compatibility layer.
|
|
| 544 |
There is one upstream provider family:
|
| 545 |
[providers/openai_chat/](src/free_claude_code/providers/openai_chat/) implements the concrete
|
| 546 |
`OpenAIChatProvider` used by every OpenAI-compatible `/chat/completions`
|
| 547 |
-
upstream. `OpenAIChatProfile` contains immutable request policy,
|
|
|
|
| 548 |
streamed-reasoning field, postprocessors, and base-URL normalization for
|
| 549 |
ordinary vendors. Configuration differences therefore remain data rather than
|
| 550 |
empty subclasses. The package also
|
|
@@ -559,7 +573,7 @@ LM Studio composes the OpenAI-chat conversion first and its context-budget probe
|
|
| 559 |
second; conversion failure therefore cannot open a stream or run the probe.
|
| 560 |
|
| 561 |
Providers call the OpenAI request policy for Anthropic-to-OpenAI conversion,
|
| 562 |
-
|
| 563 |
Specialized provider packages remain only for true upstream quirks such as
|
| 564 |
Gemini thought signatures, NIM tool-schema aliases, retry downgrades, and NVCF
|
| 565 |
deployment-failure classification, or DeepSeek attachment/tool/thinking
|
|
@@ -585,11 +599,47 @@ the GLM Coding Plan provider and uses Z.ai's Coding Plan OpenAI base.
|
|
| 585 |
Mistral La Plateforme keeps its native `reasoning_effort` and thinking-chunk
|
| 586 |
request/stream mapping inside
|
| 587 |
[providers/mistral/reasoning.py](src/free_claude_code/providers/mistral/reasoning.py), including its
|
| 588 |
-
fallback retry when
|
| 589 |
NIM reasoning budget control is also treated as a provider-owned best-effort
|
| 590 |
downgrade: if an upstream NIM deployment rejects explicit budget control, FCC
|
| 591 |
retries without the budget while preserving thinking enablement.
|
| 592 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 593 |
Shared provider responsibilities include upstream rate limiting, model listing,
|
| 594 |
SDK/HTTP failure classification, safe diagnostic construction, HTTP resource
|
| 595 |
cleanup, thinking/tool handling, retry or recovery where supported, and
|
|
@@ -755,10 +805,11 @@ tools with a single string `input` field, and restores `custom_tool_call`,
|
|
| 755 |
Responses edge. Text or grammar format metadata is preserved as model guidance;
|
| 756 |
FCC does not validate custom-tool grammars.
|
| 757 |
|
| 758 |
-
Responses reasoning is handled as protocol conversion
|
| 759 |
-
`reasoning.effort
|
| 760 |
-
|
| 761 |
-
|
|
|
|
| 762 |
Prior Responses `reasoning` input items replay plaintext `reasoning_text`, or
|
| 763 |
fallback `summary_text`, into assistant `reasoning_content`. Encrypted reasoning
|
| 764 |
input is ignored because the proxy cannot decrypt it.
|
|
@@ -794,7 +845,7 @@ handling. Each optimization is controlled by settings flags.
|
|
| 794 |
|
| 795 |
Claude Code auto-mode safety-classifier requests are a message-only routing
|
| 796 |
policy, not a short-circuit response. After routing, the Messages handler detects the
|
| 797 |
-
narrow classifier prompt shape and forces
|
| 798 |
so Claude Code receives a parser-readable `<block>yes</block>` or
|
| 799 |
`<block>no</block>` verdict.
|
| 800 |
|
|
@@ -1321,7 +1372,7 @@ Update this file when a change adds or meaningfully changes:
|
|
| 1321 |
- startup, shutdown, or resource ownership;
|
| 1322 |
- configuration precedence or managed config behavior;
|
| 1323 |
- provider runtime, catalog, or upstream-adapter architecture;
|
| 1324 |
-
- model routing or
|
| 1325 |
- CLI adapter behavior;
|
| 1326 |
- messaging platform behavior;
|
| 1327 |
- protocol conversion or streaming contracts;
|
|
|
|
| 312 |
|
| 313 |
- `MODEL` is the fallback provider-prefixed model ref.
|
| 314 |
- `MODEL_FABLE`, `MODEL_OPUS`, `MODEL_SONNET`, and `MODEL_HAIKU` override Claude model tiers.
|
| 315 |
+
- `REASONING_POLICY` selects `off`, `client`, `low`, `medium`, `high`, `xhigh`,
|
| 316 |
+
or `max` for the fallback route.
|
| 317 |
+
- `REASONING_FABLE`, `REASONING_OPUS`, `REASONING_SONNET`, and
|
| 318 |
+
`REASONING_HAIKU` accept the same values plus `inherit`.
|
| 319 |
+
|
| 320 |
+
[config/reasoning.py](src/free_claude_code/config/reasoning.py) owns the typed
|
| 321 |
+
configuration vocabulary. FCC-owned dotenv files receive a one-time rename and
|
| 322 |
+
value migration from the retired boolean settings; explicit `FCC_ENV_FILE`
|
| 323 |
+
files are never rewritten and instead receive an actionable startup warning.
|
| 324 |
|
| 325 |
[config/model_refs.py](src/free_claude_code/config/model_refs.py) owns provider-prefixed model ref
|
| 326 |
parsing and configured `MODEL*` inventory. API routing and provider validation
|
|
|
|
| 440 |
Route->>Manager: acquire current generation
|
| 441 |
Manager-->>Route: Lease(settings, provider resolver)
|
| 442 |
Route->>Handler: create message
|
| 443 |
+
Handler->>Router: resolve model and reasoning intent
|
| 444 |
Handler->>Handler: server tools or optimizations
|
| 445 |
Handler->>Exec: stream routed request
|
| 446 |
Exec->>Lease: resolve provider
|
|
|
|
| 471 |
containing `fable`, `opus`, `sonnet`, or `haiku` use the matching tier override when set,
|
| 472 |
otherwise they fall back to `MODEL`.
|
| 473 |
|
| 474 |
+
The router also selects the applicable reasoning preference. Direct provider
|
| 475 |
+
refs use the root policy; Claude tier routes use a non-inherited tier override
|
| 476 |
+
or the root fallback; the no-thinking gateway variant forces `off`.
|
| 477 |
+
[application/reasoning.py](src/free_claude_code/application/reasoning.py) then
|
| 478 |
+
combines that preference with the concrete client request exactly once. The
|
| 479 |
+
resulting `ReasoningPolicy` preserves independent control, named effort, and an
|
| 480 |
+
exact client token budget without guessing provider behavior. `ResolvedModel`
|
| 481 |
+
owns the selected route and preference; `RoutedMessagesRequest` owns the final
|
| 482 |
+
request-scoped policy passed to execution.
|
| 483 |
|
| 484 |
`GET /v1/models` advertises:
|
| 485 |
|
|
|
|
| 491 |
Provider model discovery and optional thinking metadata live in the
|
| 492 |
application-level catalog owned by `ProviderRuntimeManager`.
|
| 493 |
`ProviderModelInfo.supports_thinking` alone owns discovered per-model thinking
|
| 494 |
+
support for model-list presentation; it does not select request behavior.
|
| 495 |
+
Provider adapters must never branch on upstream model names or versions to
|
| 496 |
+
translate reasoning. The catalog is not part of an individual provider
|
| 497 |
+
generation, so a hot replacement does not erase the last useful model list.
|
| 498 |
+
Discovery failures retain prior entries.
|
| 499 |
|
| 500 |
Codex-specific model picker shaping stays out of this route. `fcc-codex` fetches
|
| 501 |
the same `/v1/models` response at launch, converts FCC gateway IDs into
|
|
|
|
| 549 |
[providers/base.py](src/free_claude_code/providers/base.py) defines provider-internal construction and lifecycle contracts:
|
| 550 |
|
| 551 |
- `ProviderConfig`: shared provider settings such as API key, base URL, rate
|
| 552 |
+
limits, timeouts, proxy, and logging flags. It is a frozen internal
|
| 553 |
value whose base URL has already been resolved from the catalog.
|
| 554 |
- `BaseProvider`: the abstract implementation base for cleanup, model listing,
|
| 555 |
explicit preflight, and `stream_response()`.
|
|
|
|
| 557 |
There is one upstream provider family:
|
| 558 |
[providers/openai_chat/](src/free_claude_code/providers/openai_chat/) implements the concrete
|
| 559 |
`OpenAIChatProvider` used by every OpenAI-compatible `/chat/completions`
|
| 560 |
+
upstream. `OpenAIChatProfile` contains immutable request policy, an explicit
|
| 561 |
+
reasoning encoder, an explicit history replay mode, its standard
|
| 562 |
streamed-reasoning field, postprocessors, and base-URL normalization for
|
| 563 |
ordinary vendors. Configuration differences therefore remain data rather than
|
| 564 |
empty subclasses. The package also
|
|
|
|
| 573 |
second; conversion failure therefore cannot open a stream or run the probe.
|
| 574 |
|
| 575 |
Providers call the OpenAI request policy for Anthropic-to-OpenAI conversion,
|
| 576 |
+
reasoning replay selection, `extra_body`, and chat-completion field normalization.
|
| 577 |
Specialized provider packages remain only for true upstream quirks such as
|
| 578 |
Gemini thought signatures, NIM tool-schema aliases, retry downgrades, and NVCF
|
| 579 |
deployment-failure classification, or DeepSeek attachment/tool/thinking
|
|
|
|
| 599 |
Mistral La Plateforme keeps its native `reasoning_effort` and thinking-chunk
|
| 600 |
request/stream mapping inside
|
| 601 |
[providers/mistral/reasoning.py](src/free_claude_code/providers/mistral/reasoning.py), including its
|
| 602 |
+
fallback retry when an upstream request rejects reasoning fields.
|
| 603 |
NIM reasoning budget control is also treated as a provider-owned best-effort
|
| 604 |
downgrade: if an upstream NIM deployment rejects explicit budget control, FCC
|
| 605 |
retries without the budget while preserving thinking enablement.
|
| 606 |
|
| 607 |
+
### Reasoning Ownership
|
| 608 |
+
|
| 609 |
+
[core/reasoning.py](src/free_claude_code/core/reasoning.py) owns the immutable,
|
| 610 |
+
provider-neutral `ReasoningPolicy`. It represents three distinct facts:
|
| 611 |
+
|
| 612 |
+
- `control`: provider default, explicitly off, or explicitly on;
|
| 613 |
+
- `effort`: the client's named effort when one was supplied;
|
| 614 |
+
- `budget_tokens`: an exact positive client budget, never a derived value.
|
| 615 |
+
|
| 616 |
+
The application layer resolves configuration and client input into this value;
|
| 617 |
+
the API layer may replace it for a product policy such as the safety classifier;
|
| 618 |
+
providers receive it unchanged. Provider adapters alone translate the subset
|
| 619 |
+
their documented wire API can represent. The shared OpenAI-chat implementation
|
| 620 |
+
uses small encoder objects for named effort, reasoning objects, thinking
|
| 621 |
+
objects, chat-template booleans, exact llama.cpp budgets, and split reasoning
|
| 622 |
+
output. Specialized providers keep only translations that cannot be expressed
|
| 623 |
+
by those encoders.
|
| 624 |
+
|
| 625 |
+
Reasoning history replay is a separate request-conversion decision. Every
|
| 626 |
+
profile explicitly chooses native `reasoning_content`, native `reasoning`,
|
| 627 |
+
`<think>` tags, provider-specific chunks, or no replay. Turning off computation
|
| 628 |
+
for the next generation does not silently erase prior assistant state required
|
| 629 |
+
for a valid continuation.
|
| 630 |
+
|
| 631 |
+
The boundary has four hard rules:
|
| 632 |
+
|
| 633 |
+
1. Never inspect an upstream model name or version to select reasoning behavior.
|
| 634 |
+
2. Never convert a named effort into a fabricated token budget, or use the
|
| 635 |
+
output-token limit as a reasoning budget.
|
| 636 |
+
3. Forward an exact token budget only where the provider documents one; otherwise
|
| 637 |
+
translate a supported named or boolean control and leave unsupported precision
|
| 638 |
+
to the provider.
|
| 639 |
+
4. Provider-default intent emits no compute-control field. Explicit off requests
|
| 640 |
+
an upstream disable where supported and always suppresses reasoning output at
|
| 641 |
+
the FCC protocol boundary.
|
| 642 |
+
|
| 643 |
Shared provider responsibilities include upstream rate limiting, model listing,
|
| 644 |
SDK/HTTP failure classification, safe diagnostic construction, HTTP resource
|
| 645 |
cleanup, thinking/tool handling, retry or recovery where supported, and
|
|
|
|
| 805 |
Responses edge. Text or grammar format metadata is preserved as model guidance;
|
| 806 |
FCC does not validate custom-tool grammars.
|
| 807 |
|
| 808 |
+
Responses reasoning is handled as lossless protocol conversion before provider
|
| 809 |
+
policy. The adapter preserves `reasoning.effort` in Anthropic `output_config`;
|
| 810 |
+
the application reasoning boundary then interprets `none` as off and preserves
|
| 811 |
+
all other named efforts. It never translates OpenAI effort names into Anthropic
|
| 812 |
+
token budgets.
|
| 813 |
Prior Responses `reasoning` input items replay plaintext `reasoning_text`, or
|
| 814 |
fallback `summary_text`, into assistant `reasoning_content`. Encrypted reasoning
|
| 815 |
input is ignored because the proxy cannot decrypt it.
|
|
|
|
| 845 |
|
| 846 |
Claude Code auto-mode safety-classifier requests are a message-only routing
|
| 847 |
policy, not a short-circuit response. After routing, the Messages handler detects the
|
| 848 |
+
narrow classifier prompt shape and forces reasoning off before provider execution
|
| 849 |
so Claude Code receives a parser-readable `<block>yes</block>` or
|
| 850 |
`<block>no</block>` verdict.
|
| 851 |
|
|
|
|
| 1372 |
- startup, shutdown, or resource ownership;
|
| 1373 |
- configuration precedence or managed config behavior;
|
| 1374 |
- provider runtime, catalog, or upstream-adapter architecture;
|
| 1375 |
+
- model routing or reasoning behavior;
|
| 1376 |
- CLI adapter behavior;
|
| 1377 |
- messaging platform behavior;
|
| 1378 |
- protocol conversion or streaming contracts;
|
|
@@ -35,6 +35,7 @@
|
|
| 35 |
- **DRY**: Extract shared base classes to eliminate duplication. Prefer composition over copy-paste.
|
| 36 |
- **Encapsulation**: Use accessor methods for internal state (e.g. `set_current_task()`), not direct `_attribute` assignment from outside.
|
| 37 |
- **Provider-specific config**: Keep provider-specific fields (e.g. `nim_settings`) in provider constructors, not in the base `ProviderConfig`.
|
|
|
|
| 38 |
- **Dead code**: Remove unused code, legacy systems, and hardcoded values. Use settings/config instead of literals (e.g. `settings.provider_type` not `"nvidia_nim"`).
|
| 39 |
- **Performance**: Use list accumulation for strings (not `+=` in loops), cache env vars at init, prefer iterative over recursive when stack depth matters.
|
| 40 |
- **Platform-agnostic naming**: Use generic names (e.g. `PLATFORM_EDIT`) not platform-specific ones (e.g. `TELEGRAM_EDIT`) in shared code.
|
|
|
|
| 35 |
- **DRY**: Extract shared base classes to eliminate duplication. Prefer composition over copy-paste.
|
| 36 |
- **Encapsulation**: Use accessor methods for internal state (e.g. `set_current_task()`), not direct `_attribute` assignment from outside.
|
| 37 |
- **Provider-specific config**: Keep provider-specific fields (e.g. `nim_settings`) in provider constructors, not in the base `ProviderConfig`.
|
| 38 |
+
- **Model-independent reasoning**: Resolve client reasoning intent once at the application boundary; provider adapters translate documented provider capabilities. Never branch on upstream model names or versions to choose reasoning behavior.
|
| 39 |
- **Dead code**: Remove unused code, legacy systems, and hardcoded values. Use settings/config instead of literals (e.g. `settings.provider_type` not `"nvidia_nim"`).
|
| 40 |
- **Performance**: Use list accumulation for strings (not `+=` in loops), cache env vars at init, prefer iterative over recursive when stack depth matters.
|
| 41 |
- **Platform-agnostic naming**: Use generic names (e.g. `PLATFORM_EDIT`) not platform-specific ones (e.g. `TELEGRAM_EDIT`) in shared code.
|
|
@@ -201,6 +201,12 @@ Use the tag shown by `ollama list` with the `ollama/` prefix. `OLLAMA_BASE_URL`
|
|
| 201 |
|
| 202 |
For example, route Opus to `nvidia_nim/moonshotai/kimi-k2.6`, Sonnet to `open_router/openrouter/free`, Haiku to `lmstudio/qwen3.5-coder`, and keep `MODEL` on `zai/glm-5.2`.
|
| 203 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 204 |
<a id="connect-your-client"></a>
|
| 205 |
|
| 206 |
## Connect Your Client
|
|
|
|
| 201 |
|
| 202 |
For example, route Opus to `nvidia_nim/moonshotai/kimi-k2.6`, Sonnet to `open_router/openrouter/free`, Haiku to `lmstudio/qwen3.5-coder`, and keep `MODEL` on `zai/glm-5.2`.
|
| 203 |
|
| 204 |
+
### Reasoning Control
|
| 205 |
+
|
| 206 |
+
Open **Admin UI → Model Config → Reasoning** to choose how FCC handles client reasoning controls. The default **From client** option preserves reasoning effort sent by Claude Code, Codex, or Pi; when the client sends no control, the provider keeps its own default.
|
| 207 |
+
|
| 208 |
+
You can instead select **Off**, **Low**, **Medium**, **High**, **X-High**, or **Max**. Fable, Opus, Sonnet, and Haiku each have the same choices plus **Inherit**, which uses the root policy. FCC translates each choice only into controls documented by that provider, so unsupported precision safely remains provider-defined.
|
| 209 |
+
|
| 210 |
<a id="connect-your-client"></a>
|
| 211 |
|
| 212 |
## Connect Your Client
|
|
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|
| 4 |
|
| 5 |
[project]
|
| 6 |
name = "free-claude-code"
|
| 7 |
-
version = "4.
|
| 8 |
description = "Local proxy connecting coding agents to OpenAI-compatible AI providers"
|
| 9 |
readme = "README.md"
|
| 10 |
requires-python = ">=3.14.0"
|
|
|
|
| 4 |
|
| 5 |
[project]
|
| 6 |
name = "free-claude-code"
|
| 7 |
+
version = "4.8.0"
|
| 8 |
description = "Local proxy connecting coding agents to OpenAI-compatible AI providers"
|
| 9 |
readme = "README.md"
|
| 10 |
requires-python = ">=3.14.0"
|
|
@@ -201,7 +201,7 @@ CAPABILITY_CONTRACTS: tuple[CapabilityContract, ...] = (
|
|
| 201 |
"tests/providers/test_open_router.py",
|
| 202 |
),
|
| 203 |
(
|
| 204 |
-
"
|
| 205 |
"test_provider_reasoning_tool_continuation_e2e",
|
| 206 |
),
|
| 207 |
),
|
|
@@ -269,10 +269,11 @@ CAPABILITY_CONTRACTS: tuple[CapabilityContract, ...] = (
|
|
| 269 |
"removed_env_migration",
|
| 270 |
"removed_env_migration",
|
| 271 |
"free_claude_code.config.settings.Settings",
|
| 272 |
-
"
|
| 273 |
-
"
|
| 274 |
-
"
|
| 275 |
-
("tests/config/
|
|
|
|
| 276 |
),
|
| 277 |
CapabilityContract(
|
| 278 |
"provider_runtime",
|
|
|
|
| 201 |
"tests/providers/test_open_router.py",
|
| 202 |
),
|
| 203 |
(
|
| 204 |
+
"test_route_reasoning_config_e2e",
|
| 205 |
"test_provider_reasoning_tool_continuation_e2e",
|
| 206 |
),
|
| 207 |
),
|
|
|
|
| 269 |
"removed_env_migration",
|
| 270 |
"removed_env_migration",
|
| 271 |
"free_claude_code.config.settings.Settings",
|
| 272 |
+
"retired ENABLE_MODEL_THINKING or tier override in an FCC-owned dotenv",
|
| 273 |
+
"owned dotenv is migrated to the typed reasoning policy",
|
| 274 |
+
"explicit dotenv receives an actionable warning",
|
| 275 |
+
("tests/config/test_env_migrations.py", "tests/config/test_config.py"),
|
| 276 |
+
("test_removed_env_migration_e2e",),
|
| 277 |
),
|
| 278 |
CapabilityContract(
|
| 279 |
"provider_runtime",
|
|
@@ -157,7 +157,7 @@ FEATURE_INVENTORY: tuple[FeatureCoverage, ...] = (
|
|
| 157 |
"test_provider_reasoning_tool_continuation_e2e",
|
| 158 |
"test_gemini_thought_signature_tool_continuation_e2e",
|
| 159 |
"test_claude_cli_adaptive_thinking_e2e",
|
| 160 |
-
"
|
| 161 |
),
|
| 162 |
("providers", "cli", "config"),
|
| 163 |
("configured provider",),
|
|
|
|
| 157 |
"test_provider_reasoning_tool_continuation_e2e",
|
| 158 |
"test_gemini_thought_signature_tool_continuation_e2e",
|
| 159 |
"test_claude_cli_adaptive_thinking_e2e",
|
| 160 |
+
"test_route_reasoning_config_e2e",
|
| 161 |
),
|
| 162 |
("providers", "cli", "config"),
|
| 163 |
("configured provider",),
|
|
@@ -64,14 +64,14 @@ def test_removed_env_migration_e2e(smoke_config: SmokeConfig, tmp_path) -> None:
|
|
| 64 |
|
| 65 |
|
| 66 |
@pytest.mark.smoke_target("config")
|
| 67 |
-
def
|
| 68 |
env_file = tmp_path / "thinking.env"
|
| 69 |
env_file.write_text(
|
| 70 |
-
'
|
| 71 |
-
'
|
| 72 |
-
'
|
| 73 |
-
|
| 74 |
-
'
|
| 75 |
encoding="utf-8",
|
| 76 |
)
|
| 77 |
env = os.environ.copy()
|
|
@@ -81,11 +81,11 @@ def test_per_model_thinking_config_e2e(smoke_config: SmokeConfig, tmp_path) -> N
|
|
| 81 |
"from free_claude_code.config.settings import Settings; "
|
| 82 |
"s=Settings(); "
|
| 83 |
"r=ModelRouter(s); "
|
| 84 |
-
"print(r.resolve('claude-fable-5').
|
| 85 |
-
"print(r.resolve('claude-opus-4-20250514').
|
| 86 |
-
"print(r.resolve('claude-sonnet-4-20250514').
|
| 87 |
-
"print(r.resolve('claude-haiku-4-20250514').
|
| 88 |
-
"print(r.resolve('unknown-model').
|
| 89 |
)
|
| 90 |
result = run_captured_text(
|
| 91 |
cmd_python_c(script),
|
|
@@ -96,11 +96,11 @@ def test_per_model_thinking_config_e2e(smoke_config: SmokeConfig, tmp_path) -> N
|
|
| 96 |
)
|
| 97 |
assert result.returncode == 0, result.stderr
|
| 98 |
assert result.stdout.splitlines() == [
|
| 99 |
-
"
|
| 100 |
-
"
|
| 101 |
-
"
|
| 102 |
-
"
|
| 103 |
-
"
|
| 104 |
]
|
| 105 |
|
| 106 |
|
|
|
|
| 64 |
|
| 65 |
|
| 66 |
@pytest.mark.smoke_target("config")
|
| 67 |
+
def test_route_reasoning_config_e2e(smoke_config: SmokeConfig, tmp_path) -> None:
|
| 68 |
env_file = tmp_path / "thinking.env"
|
| 69 |
env_file.write_text(
|
| 70 |
+
'REASONING_POLICY="off"\n'
|
| 71 |
+
'REASONING_FABLE="high"\n'
|
| 72 |
+
'REASONING_OPUS="client"\n'
|
| 73 |
+
'REASONING_SONNET="inherit"\n'
|
| 74 |
+
'REASONING_HAIKU="off"\n',
|
| 75 |
encoding="utf-8",
|
| 76 |
)
|
| 77 |
env = os.environ.copy()
|
|
|
|
| 81 |
"from free_claude_code.config.settings import Settings; "
|
| 82 |
"s=Settings(); "
|
| 83 |
"r=ModelRouter(s); "
|
| 84 |
+
"print(r.resolve('claude-fable-5').reasoning_preference.value); "
|
| 85 |
+
"print(r.resolve('claude-opus-4-20250514').reasoning_preference.value); "
|
| 86 |
+
"print(r.resolve('claude-sonnet-4-20250514').reasoning_preference.value); "
|
| 87 |
+
"print(r.resolve('claude-haiku-4-20250514').reasoning_preference.value); "
|
| 88 |
+
"print(r.resolve('unknown-model').reasoning_preference.value)"
|
| 89 |
)
|
| 90 |
result = run_captured_text(
|
| 91 |
cmd_python_c(script),
|
|
|
|
| 96 |
)
|
| 97 |
assert result.returncode == 0, result.stderr
|
| 98 |
assert result.stdout.splitlines() == [
|
| 99 |
+
"high",
|
| 100 |
+
"client",
|
| 101 |
+
"off",
|
| 102 |
+
"off",
|
| 103 |
+
"off",
|
| 104 |
]
|
| 105 |
|
| 106 |
|
|
@@ -35,7 +35,7 @@ def test_nvidia_nim_cli_matrix_e2e(smoke_config: SmokeConfig, tmp_path: Path) ->
|
|
| 35 |
env_overrides={
|
| 36 |
"MODEL": provider_model.full_model,
|
| 37 |
"MESSAGING_PLATFORM": "none",
|
| 38 |
-
"
|
| 39 |
"LOG_RAW_API_PAYLOADS": "true",
|
| 40 |
"LOG_RAW_SSE_EVENTS": "true",
|
| 41 |
},
|
|
|
|
| 35 |
env_overrides={
|
| 36 |
"MODEL": provider_model.full_model,
|
| 37 |
"MESSAGING_PLATFORM": "none",
|
| 38 |
+
"REASONING_POLICY": "high",
|
| 39 |
"LOG_RAW_API_PAYLOADS": "true",
|
| 40 |
"LOG_RAW_SSE_EVENTS": "true",
|
| 41 |
},
|
|
@@ -37,7 +37,7 @@ def test_openrouter_free_cli_matrix_e2e(
|
|
| 37 |
env_overrides={
|
| 38 |
"MODEL": provider_model.full_model,
|
| 39 |
"MESSAGING_PLATFORM": "none",
|
| 40 |
-
"
|
| 41 |
"LOG_RAW_API_PAYLOADS": "true",
|
| 42 |
"LOG_RAW_SSE_EVENTS": "true",
|
| 43 |
},
|
|
|
|
| 37 |
env_overrides={
|
| 38 |
"MODEL": provider_model.full_model,
|
| 39 |
"MESSAGING_PLATFORM": "none",
|
| 40 |
+
"REASONING_POLICY": "high",
|
| 41 |
"LOG_RAW_API_PAYLOADS": "true",
|
| 42 |
"LOG_RAW_SSE_EVENTS": "true",
|
| 43 |
},
|
|
@@ -4,6 +4,7 @@ import httpx
|
|
| 4 |
import pytest
|
| 5 |
|
| 6 |
from free_claude_code.application.routing import ModelRouter
|
|
|
|
| 7 |
from free_claude_code.core.anthropic.stream_contracts import (
|
| 8 |
SSEEvent,
|
| 9 |
parse_sse_lines,
|
|
@@ -101,8 +102,8 @@ def test_gemini_thought_signature_tool_continuation_e2e(
|
|
| 101 |
def test_provider_reasoning_tool_continuation_e2e(
|
| 102 |
smoke_config: SmokeConfig, provider_model: ProviderModel
|
| 103 |
) -> None:
|
| 104 |
-
if not
|
| 105 |
-
pytest.skip("the configured Claude route
|
| 106 |
_run_provider_scenario(
|
| 107 |
smoke_config, provider_model, _scenario_reasoning_tool_continuation
|
| 108 |
)
|
|
@@ -267,11 +268,12 @@ def _tool_use_blocks_or_skip(
|
|
| 267 |
return blocks
|
| 268 |
|
| 269 |
|
| 270 |
-
def
|
| 271 |
return (
|
| 272 |
ModelRouter(smoke_config.settings)
|
| 273 |
.resolve("claude-sonnet-4-5-20250929")
|
| 274 |
-
.
|
|
|
|
| 275 |
)
|
| 276 |
|
| 277 |
|
|
|
|
| 4 |
import pytest
|
| 5 |
|
| 6 |
from free_claude_code.application.routing import ModelRouter
|
| 7 |
+
from free_claude_code.config.reasoning import ReasoningPreference
|
| 8 |
from free_claude_code.core.anthropic.stream_contracts import (
|
| 9 |
SSEEvent,
|
| 10 |
parse_sse_lines,
|
|
|
|
| 102 |
def test_provider_reasoning_tool_continuation_e2e(
|
| 103 |
smoke_config: SmokeConfig, provider_model: ProviderModel
|
| 104 |
) -> None:
|
| 105 |
+
if not _provider_smoke_reasoning_enabled(smoke_config):
|
| 106 |
+
pytest.skip("the configured Claude route disables reasoning")
|
| 107 |
_run_provider_scenario(
|
| 108 |
smoke_config, provider_model, _scenario_reasoning_tool_continuation
|
| 109 |
)
|
|
|
|
| 268 |
return blocks
|
| 269 |
|
| 270 |
|
| 271 |
+
def _provider_smoke_reasoning_enabled(smoke_config: SmokeConfig) -> bool:
|
| 272 |
return (
|
| 273 |
ModelRouter(smoke_config.settings)
|
| 274 |
.resolve("claude-sonnet-4-5-20250929")
|
| 275 |
+
.reasoning_preference
|
| 276 |
+
is not ReasoningPreference.OFF
|
| 277 |
)
|
| 278 |
|
| 279 |
|
|
@@ -173,8 +173,21 @@ def _message_payload(*, stream: bool) -> dict[str, Any]:
|
|
| 173 |
def _write_initial_managed_config(home: Path, upstream: FakeOpenAIUpstream) -> None:
|
| 174 |
config_path = home / ".fcc" / ".env"
|
| 175 |
config_path.parent.mkdir(parents=True, exist_ok=True)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 176 |
config_path.write_text(
|
| 177 |
-
|
| 178 |
+ "\n"
|
| 179 |
+ "\n".join(
|
| 180 |
[
|
|
@@ -245,10 +258,19 @@ def test_provider_hot_swap_preserves_inflight_stream_e2e(
|
|
| 245 |
"MODEL_HAIKU",
|
| 246 |
"MODEL_OPUS",
|
| 247 |
"MODEL_SONNET",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 248 |
},
|
| 249 |
)
|
| 250 |
)
|
| 251 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 252 |
def consume_old_stream() -> None:
|
| 253 |
try:
|
| 254 |
with httpx.stream(
|
|
|
|
| 173 |
def _write_initial_managed_config(home: Path, upstream: FakeOpenAIUpstream) -> None:
|
| 174 |
config_path = home / ".fcc" / ".env"
|
| 175 |
config_path.parent.mkdir(parents=True, exist_ok=True)
|
| 176 |
+
template = "\n".join(
|
| 177 |
+
line
|
| 178 |
+
for line in load_env_template().splitlines()
|
| 179 |
+
if not line.startswith(
|
| 180 |
+
(
|
| 181 |
+
"REASONING_POLICY=",
|
| 182 |
+
"REASONING_FABLE=",
|
| 183 |
+
"REASONING_OPUS=",
|
| 184 |
+
"REASONING_SONNET=",
|
| 185 |
+
"REASONING_HAIKU=",
|
| 186 |
+
)
|
| 187 |
+
)
|
| 188 |
+
)
|
| 189 |
config_path.write_text(
|
| 190 |
+
template
|
| 191 |
+ "\n"
|
| 192 |
+ "\n".join(
|
| 193 |
[
|
|
|
|
| 258 |
"MODEL_HAIKU",
|
| 259 |
"MODEL_OPUS",
|
| 260 |
"MODEL_SONNET",
|
| 261 |
+
"REASONING_FABLE",
|
| 262 |
+
"REASONING_HAIKU",
|
| 263 |
+
"REASONING_OPUS",
|
| 264 |
+
"REASONING_POLICY",
|
| 265 |
+
"REASONING_SONNET",
|
| 266 |
},
|
| 267 |
)
|
| 268 |
)
|
| 269 |
|
| 270 |
+
managed_config = (home / ".fcc" / ".env").read_text(encoding="utf-8")
|
| 271 |
+
assert "REASONING_POLICY=off" in managed_config
|
| 272 |
+
assert "ENABLE_MODEL_THINKING" not in managed_config
|
| 273 |
+
|
| 274 |
def consume_old_stream() -> None:
|
| 275 |
try:
|
| 276 |
with httpx.stream(
|
|
@@ -20,7 +20,7 @@ const VIEW_GROUPS = [
|
|
| 20 |
id: "model_config",
|
| 21 |
label: "Model Config",
|
| 22 |
title: "Model Config",
|
| 23 |
-
sections: ["models", "
|
| 24 |
containerId: "modelConfigSections",
|
| 25 |
},
|
| 26 |
{
|
|
@@ -308,21 +308,12 @@ function inputForField(field) {
|
|
| 308 |
return input;
|
| 309 |
}
|
| 310 |
|
| 311 |
-
if (field.type === "tri_boolean") {
|
| 312 |
-
const select = document.createElement("select");
|
| 313 |
-
[
|
| 314 |
-
["", "Inherit"],
|
| 315 |
-
["true", "Enabled"],
|
| 316 |
-
["false", "Disabled"],
|
| 317 |
-
].forEach(([value, label]) => select.appendChild(option(value, label)));
|
| 318 |
-
select.value = field.value || "";
|
| 319 |
-
return select;
|
| 320 |
-
}
|
| 321 |
-
|
| 322 |
if (field.type === "select") {
|
| 323 |
const select = document.createElement("select");
|
| 324 |
-
field.options.forEach((
|
| 325 |
-
|
|
|
|
|
|
|
| 326 |
return select;
|
| 327 |
}
|
| 328 |
|
|
|
|
| 20 |
id: "model_config",
|
| 21 |
label: "Model Config",
|
| 22 |
title: "Model Config",
|
| 23 |
+
sections: ["models", "reasoning", "web_tools"],
|
| 24 |
containerId: "modelConfigSections",
|
| 25 |
},
|
| 26 |
{
|
|
|
|
| 308 |
return input;
|
| 309 |
}
|
| 310 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 311 |
if (field.type === "select") {
|
| 312 |
const select = document.createElement("select");
|
| 313 |
+
field.options.forEach((item) =>
|
| 314 |
+
select.appendChild(option(item.value, item.label)),
|
| 315 |
+
);
|
| 316 |
+
select.value = field.value || field.options[0]?.value || "";
|
| 317 |
return select;
|
| 318 |
}
|
| 319 |
|
|
@@ -47,6 +47,7 @@ from free_claude_code.core.anthropic import (
|
|
| 47 |
)
|
| 48 |
from free_claude_code.core.diagnostics import safe_exception_message
|
| 49 |
from free_claude_code.core.failures import ExecutionFailure, find_execution_failure
|
|
|
|
| 50 |
from free_claude_code.core.trace import trace_event
|
| 51 |
|
| 52 |
|
|
@@ -271,7 +272,7 @@ class MessagesHandler:
|
|
| 271 |
) -> RoutedMessagesRequest:
|
| 272 |
if not is_safety_classifier_request(routed.request):
|
| 273 |
return routed
|
| 274 |
-
changed = routed.
|
| 275 |
trace_event(
|
| 276 |
stage="routing",
|
| 277 |
event="free_claude_code.api.optimization.safety_classifier_no_thinking",
|
|
@@ -281,10 +282,7 @@ class MessagesHandler:
|
|
| 281 |
)
|
| 282 |
if not changed:
|
| 283 |
return routed
|
| 284 |
-
return
|
| 285 |
-
request=routed.request,
|
| 286 |
-
resolved=replace(routed.resolved, thinking_enabled=False),
|
| 287 |
-
)
|
| 288 |
|
| 289 |
def _run_message_intercepts(
|
| 290 |
self, routed: RoutedMessagesRequest
|
|
|
|
| 47 |
)
|
| 48 |
from free_claude_code.core.diagnostics import safe_exception_message
|
| 49 |
from free_claude_code.core.failures import ExecutionFailure, find_execution_failure
|
| 50 |
+
from free_claude_code.core.reasoning import ReasoningControl, ReasoningPolicy
|
| 51 |
from free_claude_code.core.trace import trace_event
|
| 52 |
|
| 53 |
|
|
|
|
| 272 |
) -> RoutedMessagesRequest:
|
| 273 |
if not is_safety_classifier_request(routed.request):
|
| 274 |
return routed
|
| 275 |
+
changed = routed.reasoning.control is not ReasoningControl.OFF
|
| 276 |
trace_event(
|
| 277 |
stage="routing",
|
| 278 |
event="free_claude_code.api.optimization.safety_classifier_no_thinking",
|
|
|
|
| 282 |
)
|
| 283 |
if not changed:
|
| 284 |
return routed
|
| 285 |
+
return replace(routed, reasoning=ReasoningPolicy.off())
|
|
|
|
|
|
|
|
|
|
| 286 |
|
| 287 |
def _run_message_intercepts(
|
| 288 |
self, routed: RoutedMessagesRequest
|
|
@@ -58,7 +58,7 @@ class ProviderExecutor:
|
|
| 58 |
provider = self._provider_resolver(routed.resolved.provider_id)
|
| 59 |
provider.preflight_stream(
|
| 60 |
routed.request,
|
| 61 |
-
|
| 62 |
)
|
| 63 |
|
| 64 |
route_trace: dict[str, object] = {
|
|
@@ -70,7 +70,13 @@ class ProviderExecutor:
|
|
| 70 |
"provider_model": routed.resolved.provider_model,
|
| 71 |
"provider_model_ref": routed.resolved.provider_model_ref,
|
| 72 |
"gateway_model": routed.request.model,
|
| 73 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
}
|
| 75 |
if wire_api == "responses":
|
| 76 |
route_trace["wire_api"] = "responses"
|
|
@@ -107,7 +113,7 @@ class ProviderExecutor:
|
|
| 107 |
routed.request,
|
| 108 |
input_tokens=input_tokens,
|
| 109 |
request_id=request_id,
|
| 110 |
-
|
| 111 |
)
|
| 112 |
async for chunk in provider_stream:
|
| 113 |
yield chunk
|
|
|
|
| 58 |
provider = self._provider_resolver(routed.resolved.provider_id)
|
| 59 |
provider.preflight_stream(
|
| 60 |
routed.request,
|
| 61 |
+
reasoning=routed.reasoning,
|
| 62 |
)
|
| 63 |
|
| 64 |
route_trace: dict[str, object] = {
|
|
|
|
| 70 |
"provider_model": routed.resolved.provider_model,
|
| 71 |
"provider_model_ref": routed.resolved.provider_model_ref,
|
| 72 |
"gateway_model": routed.request.model,
|
| 73 |
+
"reasoning_control": routed.reasoning.control.value,
|
| 74 |
+
"reasoning_effort": (
|
| 75 |
+
routed.reasoning.effort.value
|
| 76 |
+
if routed.reasoning.effort is not None
|
| 77 |
+
else None
|
| 78 |
+
),
|
| 79 |
+
"reasoning_budget_tokens": routed.reasoning.budget_tokens,
|
| 80 |
}
|
| 81 |
if wire_api == "responses":
|
| 82 |
route_trace["wire_api"] = "responses"
|
|
|
|
| 113 |
routed.request,
|
| 114 |
input_tokens=input_tokens,
|
| 115 |
request_id=request_id,
|
| 116 |
+
reasoning=routed.reasoning,
|
| 117 |
)
|
| 118 |
async for chunk in provider_stream:
|
| 119 |
yield chunk
|
|
@@ -6,6 +6,7 @@ from typing import Protocol
|
|
| 6 |
|
| 7 |
from free_claude_code.config.settings import Settings
|
| 8 |
from free_claude_code.core.anthropic import MessagesRequest
|
|
|
|
| 9 |
|
| 10 |
from .model_metadata import ProviderModelInfo
|
| 11 |
|
|
@@ -17,7 +18,7 @@ class ProviderPort(Protocol):
|
|
| 17 |
self,
|
| 18 |
request: MessagesRequest,
|
| 19 |
*,
|
| 20 |
-
|
| 21 |
) -> None: ...
|
| 22 |
|
| 23 |
def stream_response(
|
|
@@ -26,7 +27,7 @@ class ProviderPort(Protocol):
|
|
| 26 |
*,
|
| 27 |
input_tokens: int,
|
| 28 |
request_id: str,
|
| 29 |
-
|
| 30 |
) -> AsyncIterator[str]: ...
|
| 31 |
|
| 32 |
|
|
|
|
| 6 |
|
| 7 |
from free_claude_code.config.settings import Settings
|
| 8 |
from free_claude_code.core.anthropic import MessagesRequest
|
| 9 |
+
from free_claude_code.core.reasoning import ReasoningPolicy
|
| 10 |
|
| 11 |
from .model_metadata import ProviderModelInfo
|
| 12 |
|
|
|
|
| 18 |
self,
|
| 19 |
request: MessagesRequest,
|
| 20 |
*,
|
| 21 |
+
reasoning: ReasoningPolicy,
|
| 22 |
) -> None: ...
|
| 23 |
|
| 24 |
def stream_response(
|
|
|
|
| 27 |
*,
|
| 28 |
input_tokens: int,
|
| 29 |
request_id: str,
|
| 30 |
+
reasoning: ReasoningPolicy,
|
| 31 |
) -> AsyncIterator[str]: ...
|
| 32 |
|
| 33 |
|
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Resolve client reasoning input and FCC configuration exactly once."""
|
| 2 |
+
|
| 3 |
+
from collections.abc import Mapping
|
| 4 |
+
from typing import Any
|
| 5 |
+
|
| 6 |
+
from free_claude_code.config.reasoning import ReasoningPreference
|
| 7 |
+
from free_claude_code.core.anthropic.models import MessagesRequest, ThinkingConfig
|
| 8 |
+
from free_claude_code.core.reasoning import (
|
| 9 |
+
ReasoningControl,
|
| 10 |
+
ReasoningEffort,
|
| 11 |
+
ReasoningPolicy,
|
| 12 |
+
)
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
def resolve_reasoning_policy(
|
| 16 |
+
request: MessagesRequest,
|
| 17 |
+
preference: ReasoningPreference,
|
| 18 |
+
) -> ReasoningPolicy:
|
| 19 |
+
"""Apply one resolved configuration preference to the client request."""
|
| 20 |
+
|
| 21 |
+
if preference is ReasoningPreference.INHERIT:
|
| 22 |
+
raise ValueError("Reasoning preference must be resolved before application.")
|
| 23 |
+
if preference is ReasoningPreference.OFF:
|
| 24 |
+
return ReasoningPolicy.off()
|
| 25 |
+
if preference is not ReasoningPreference.CLIENT:
|
| 26 |
+
return ReasoningPolicy.on(effort=ReasoningEffort(preference.value))
|
| 27 |
+
return client_reasoning_policy(request)
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
def client_reasoning_policy(request: MessagesRequest) -> ReasoningPolicy:
|
| 31 |
+
"""Return the lossless reasoning intent expressed by one client request."""
|
| 32 |
+
|
| 33 |
+
budget_tokens = _positive_budget(request.thinking)
|
| 34 |
+
thinking_control = _thinking_control(
|
| 35 |
+
request.thinking,
|
| 36 |
+
budget_tokens=budget_tokens,
|
| 37 |
+
)
|
| 38 |
+
effort, effort_disables = _output_effort(request.output_config)
|
| 39 |
+
|
| 40 |
+
if effort_disables:
|
| 41 |
+
return ReasoningPolicy.off()
|
| 42 |
+
if thinking_control is ReasoningControl.OFF:
|
| 43 |
+
return ReasoningPolicy(
|
| 44 |
+
control=ReasoningControl.OFF,
|
| 45 |
+
effort=effort,
|
| 46 |
+
)
|
| 47 |
+
if thinking_control is ReasoningControl.ON or budget_tokens is not None:
|
| 48 |
+
return ReasoningPolicy.on(
|
| 49 |
+
effort=effort,
|
| 50 |
+
budget_tokens=budget_tokens,
|
| 51 |
+
)
|
| 52 |
+
return ReasoningPolicy(
|
| 53 |
+
control=ReasoningControl.DEFAULT,
|
| 54 |
+
effort=effort,
|
| 55 |
+
)
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
def _thinking_control(
|
| 59 |
+
thinking: ThinkingConfig | None,
|
| 60 |
+
*,
|
| 61 |
+
budget_tokens: int | None,
|
| 62 |
+
) -> ReasoningControl:
|
| 63 |
+
if thinking is None:
|
| 64 |
+
return ReasoningControl.DEFAULT
|
| 65 |
+
if thinking.type == "disabled" or (
|
| 66 |
+
"enabled" in thinking.model_fields_set and thinking.enabled is False
|
| 67 |
+
):
|
| 68 |
+
return ReasoningControl.OFF
|
| 69 |
+
if (
|
| 70 |
+
thinking.type in {"adaptive", "enabled"}
|
| 71 |
+
or ("enabled" in thinking.model_fields_set and thinking.enabled is True)
|
| 72 |
+
or budget_tokens is not None
|
| 73 |
+
):
|
| 74 |
+
return ReasoningControl.ON
|
| 75 |
+
return ReasoningControl.DEFAULT
|
| 76 |
+
|
| 77 |
+
|
| 78 |
+
def _output_effort(value: Any) -> tuple[ReasoningEffort | None, bool]:
|
| 79 |
+
if not isinstance(value, Mapping):
|
| 80 |
+
return None, False
|
| 81 |
+
raw = value.get("effort")
|
| 82 |
+
if not isinstance(raw, str):
|
| 83 |
+
return None, False
|
| 84 |
+
normalized = raw.strip().lower()
|
| 85 |
+
if normalized == "none":
|
| 86 |
+
return None, True
|
| 87 |
+
try:
|
| 88 |
+
return ReasoningEffort(normalized), False
|
| 89 |
+
except ValueError:
|
| 90 |
+
return None, False
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
def _positive_budget(thinking: ThinkingConfig | None) -> int | None:
|
| 94 |
+
if thinking is None:
|
| 95 |
+
return None
|
| 96 |
+
value = thinking.budget_tokens
|
| 97 |
+
if isinstance(value, int) and not isinstance(value, bool) and value > 0:
|
| 98 |
+
return value
|
| 99 |
+
return None
|
|
@@ -10,9 +10,20 @@ from free_claude_code.config.provider_catalog import (
|
|
| 10 |
PROVIDER_CATALOG,
|
| 11 |
SUPPORTED_PROVIDER_IDS,
|
| 12 |
)
|
|
|
|
| 13 |
from free_claude_code.config.settings import Settings
|
| 14 |
from free_claude_code.core.anthropic import MessagesRequest, TokenCountRequest
|
| 15 |
from free_claude_code.core.gateway_model_ids import decode_gateway_model_id
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
|
| 18 |
@dataclass(frozen=True, slots=True)
|
|
@@ -21,13 +32,14 @@ class ResolvedModel:
|
|
| 21 |
provider_id: str
|
| 22 |
provider_model: str
|
| 23 |
provider_model_ref: str
|
| 24 |
-
|
| 25 |
|
| 26 |
|
| 27 |
@dataclass(frozen=True, slots=True)
|
| 28 |
class RoutedMessagesRequest:
|
| 29 |
request: MessagesRequest
|
| 30 |
resolved: ResolvedModel
|
|
|
|
| 31 |
|
| 32 |
|
| 33 |
@dataclass(frozen=True, slots=True)
|
|
@@ -46,31 +58,31 @@ class ModelRouter:
|
|
| 46 |
(
|
| 47 |
direct_provider_id,
|
| 48 |
direct_provider_model,
|
| 49 |
-
|
| 50 |
) = self._direct_provider_model(claude_model_name)
|
| 51 |
if direct_provider_id is not None and direct_provider_model is not None:
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
if
|
| 55 |
-
else self.
|
| 56 |
)
|
| 57 |
logger.debug(
|
| 58 |
-
"MODEL DIRECT: '{}' -> provider='{}' model='{}'
|
| 59 |
claude_model_name,
|
| 60 |
direct_provider_id,
|
| 61 |
direct_provider_model,
|
| 62 |
-
|
| 63 |
)
|
| 64 |
return ResolvedModel(
|
| 65 |
original_model=claude_model_name,
|
| 66 |
provider_id=direct_provider_id,
|
| 67 |
provider_model=direct_provider_model,
|
| 68 |
provider_model_ref=claude_model_name,
|
| 69 |
-
|
| 70 |
)
|
| 71 |
|
| 72 |
provider_model_ref = self._resolve_model_ref(claude_model_name)
|
| 73 |
-
|
| 74 |
provider_id = parse_provider_type(provider_model_ref)
|
| 75 |
self._validate_provider_id(provider_id)
|
| 76 |
provider_model = parse_model_name(provider_model_ref)
|
|
@@ -83,7 +95,7 @@ class ModelRouter:
|
|
| 83 |
provider_id=provider_id,
|
| 84 |
provider_model=provider_model,
|
| 85 |
provider_model_ref=provider_model_ref,
|
| 86 |
-
|
| 87 |
)
|
| 88 |
|
| 89 |
@staticmethod
|
|
@@ -93,53 +105,55 @@ class ModelRouter:
|
|
| 93 |
|
| 94 |
def _direct_provider_model(
|
| 95 |
self, model_name: str
|
| 96 |
-
) -> tuple[str | None, str | None, bool
|
| 97 |
decoded = decode_gateway_model_id(model_name)
|
| 98 |
if decoded is not None:
|
| 99 |
if decoded.provider_id not in SUPPORTED_PROVIDER_IDS:
|
| 100 |
-
return None, None,
|
| 101 |
return (
|
| 102 |
decoded.provider_id,
|
| 103 |
decoded.provider_model,
|
| 104 |
-
decoded.
|
| 105 |
)
|
| 106 |
|
| 107 |
provider_id, separator, provider_model = model_name.partition("/")
|
| 108 |
if not separator:
|
| 109 |
-
return None, None,
|
| 110 |
if provider_id not in SUPPORTED_PROVIDER_IDS:
|
| 111 |
-
return None, None,
|
| 112 |
if not provider_model:
|
| 113 |
-
return None, None,
|
| 114 |
-
return provider_id, provider_model,
|
| 115 |
|
| 116 |
def _resolve_model_ref(self, claude_model_name: str) -> str:
|
| 117 |
"""Resolve a Claude model name to the configured provider/model ref."""
|
| 118 |
|
| 119 |
-
|
| 120 |
-
if
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
if "haiku" in name_lower and self._settings.model_haiku is not None:
|
| 125 |
-
return self._settings.model_haiku
|
| 126 |
-
if "sonnet" in name_lower and self._settings.model_sonnet is not None:
|
| 127 |
-
return self._settings.model_sonnet
|
| 128 |
return self._settings.model
|
| 129 |
|
| 130 |
-
def
|
| 131 |
-
|
|
|
|
|
|
|
| 132 |
|
| 133 |
-
|
| 134 |
-
if
|
| 135 |
-
|
| 136 |
-
|
| 137 |
-
|
| 138 |
-
|
| 139 |
-
|
| 140 |
-
|
| 141 |
-
|
| 142 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 143 |
|
| 144 |
def resolve_messages_request(
|
| 145 |
self, request: MessagesRequest
|
|
@@ -148,7 +162,14 @@ class ModelRouter:
|
|
| 148 |
resolved = self.resolve(request.model)
|
| 149 |
routed = request.model_copy(deep=True)
|
| 150 |
routed.model = resolved.provider_model
|
| 151 |
-
return RoutedMessagesRequest(
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 152 |
|
| 153 |
def resolve_token_count_request(
|
| 154 |
self, request: TokenCountRequest
|
|
|
|
| 10 |
PROVIDER_CATALOG,
|
| 11 |
SUPPORTED_PROVIDER_IDS,
|
| 12 |
)
|
| 13 |
+
from free_claude_code.config.reasoning import ReasoningPreference
|
| 14 |
from free_claude_code.config.settings import Settings
|
| 15 |
from free_claude_code.core.anthropic import MessagesRequest, TokenCountRequest
|
| 16 |
from free_claude_code.core.gateway_model_ids import decode_gateway_model_id
|
| 17 |
+
from free_claude_code.core.reasoning import ReasoningPolicy
|
| 18 |
+
|
| 19 |
+
from .reasoning import resolve_reasoning_policy
|
| 20 |
+
|
| 21 |
+
_ROUTE_SETTINGS = (
|
| 22 |
+
("fable", "model_fable", "reasoning_fable"),
|
| 23 |
+
("opus", "model_opus", "reasoning_opus"),
|
| 24 |
+
("haiku", "model_haiku", "reasoning_haiku"),
|
| 25 |
+
("sonnet", "model_sonnet", "reasoning_sonnet"),
|
| 26 |
+
)
|
| 27 |
|
| 28 |
|
| 29 |
@dataclass(frozen=True, slots=True)
|
|
|
|
| 32 |
provider_id: str
|
| 33 |
provider_model: str
|
| 34 |
provider_model_ref: str
|
| 35 |
+
reasoning_preference: ReasoningPreference
|
| 36 |
|
| 37 |
|
| 38 |
@dataclass(frozen=True, slots=True)
|
| 39 |
class RoutedMessagesRequest:
|
| 40 |
request: MessagesRequest
|
| 41 |
resolved: ResolvedModel
|
| 42 |
+
reasoning: ReasoningPolicy
|
| 43 |
|
| 44 |
|
| 45 |
@dataclass(frozen=True, slots=True)
|
|
|
|
| 58 |
(
|
| 59 |
direct_provider_id,
|
| 60 |
direct_provider_model,
|
| 61 |
+
force_reasoning_off,
|
| 62 |
) = self._direct_provider_model(claude_model_name)
|
| 63 |
if direct_provider_id is not None and direct_provider_model is not None:
|
| 64 |
+
reasoning_preference = (
|
| 65 |
+
ReasoningPreference.OFF
|
| 66 |
+
if force_reasoning_off
|
| 67 |
+
else self._settings.reasoning_policy
|
| 68 |
)
|
| 69 |
logger.debug(
|
| 70 |
+
"MODEL DIRECT: '{}' -> provider='{}' model='{}' reasoning={}",
|
| 71 |
claude_model_name,
|
| 72 |
direct_provider_id,
|
| 73 |
direct_provider_model,
|
| 74 |
+
reasoning_preference.value,
|
| 75 |
)
|
| 76 |
return ResolvedModel(
|
| 77 |
original_model=claude_model_name,
|
| 78 |
provider_id=direct_provider_id,
|
| 79 |
provider_model=direct_provider_model,
|
| 80 |
provider_model_ref=claude_model_name,
|
| 81 |
+
reasoning_preference=reasoning_preference,
|
| 82 |
)
|
| 83 |
|
| 84 |
provider_model_ref = self._resolve_model_ref(claude_model_name)
|
| 85 |
+
reasoning_preference = self._resolve_reasoning_preference(claude_model_name)
|
| 86 |
provider_id = parse_provider_type(provider_model_ref)
|
| 87 |
self._validate_provider_id(provider_id)
|
| 88 |
provider_model = parse_model_name(provider_model_ref)
|
|
|
|
| 95 |
provider_id=provider_id,
|
| 96 |
provider_model=provider_model,
|
| 97 |
provider_model_ref=provider_model_ref,
|
| 98 |
+
reasoning_preference=reasoning_preference,
|
| 99 |
)
|
| 100 |
|
| 101 |
@staticmethod
|
|
|
|
| 105 |
|
| 106 |
def _direct_provider_model(
|
| 107 |
self, model_name: str
|
| 108 |
+
) -> tuple[str | None, str | None, bool]:
|
| 109 |
decoded = decode_gateway_model_id(model_name)
|
| 110 |
if decoded is not None:
|
| 111 |
if decoded.provider_id not in SUPPORTED_PROVIDER_IDS:
|
| 112 |
+
return None, None, False
|
| 113 |
return (
|
| 114 |
decoded.provider_id,
|
| 115 |
decoded.provider_model,
|
| 116 |
+
decoded.force_reasoning_off,
|
| 117 |
)
|
| 118 |
|
| 119 |
provider_id, separator, provider_model = model_name.partition("/")
|
| 120 |
if not separator:
|
| 121 |
+
return None, None, False
|
| 122 |
if provider_id not in SUPPORTED_PROVIDER_IDS:
|
| 123 |
+
return None, None, False
|
| 124 |
if not provider_model:
|
| 125 |
+
return None, None, False
|
| 126 |
+
return provider_id, provider_model, False
|
| 127 |
|
| 128 |
def _resolve_model_ref(self, claude_model_name: str) -> str:
|
| 129 |
"""Resolve a Claude model name to the configured provider/model ref."""
|
| 130 |
|
| 131 |
+
route = self._matched_route(claude_model_name)
|
| 132 |
+
if route is not None:
|
| 133 |
+
model = getattr(self._settings, route[1])
|
| 134 |
+
if isinstance(model, str):
|
| 135 |
+
return model
|
|
|
|
|
|
|
|
|
|
|
|
|
| 136 |
return self._settings.model
|
| 137 |
|
| 138 |
+
def _resolve_reasoning_preference(
|
| 139 |
+
self, claude_model_name: str
|
| 140 |
+
) -> ReasoningPreference:
|
| 141 |
+
"""Resolve a route override without inspecting the provider model."""
|
| 142 |
|
| 143 |
+
route = self._matched_route(claude_model_name)
|
| 144 |
+
if route is not None:
|
| 145 |
+
preference = getattr(self._settings, route[2])
|
| 146 |
+
if preference is not ReasoningPreference.INHERIT:
|
| 147 |
+
return preference
|
| 148 |
+
return self._settings.reasoning_policy
|
| 149 |
+
|
| 150 |
+
@staticmethod
|
| 151 |
+
def _matched_route(model_name: str) -> tuple[str, str, str] | None:
|
| 152 |
+
normalized = model_name.lower()
|
| 153 |
+
return next(
|
| 154 |
+
(route for route in _ROUTE_SETTINGS if route[0] in normalized),
|
| 155 |
+
None,
|
| 156 |
+
)
|
| 157 |
|
| 158 |
def resolve_messages_request(
|
| 159 |
self, request: MessagesRequest
|
|
|
|
| 162 |
resolved = self.resolve(request.model)
|
| 163 |
routed = request.model_copy(deep=True)
|
| 164 |
routed.model = resolved.provider_model
|
| 165 |
+
return RoutedMessagesRequest(
|
| 166 |
+
request=routed,
|
| 167 |
+
resolved=resolved,
|
| 168 |
+
reasoning=resolve_reasoning_policy(
|
| 169 |
+
routed,
|
| 170 |
+
resolved.reasoning_preference,
|
| 171 |
+
),
|
| 172 |
+
)
|
| 173 |
|
| 174 |
def resolve_token_count_request(
|
| 175 |
self, request: TokenCountRequest
|
|
@@ -13,7 +13,7 @@ import uvicorn
|
|
| 13 |
from free_claude_code.cli.launchers.common import preflight_proxy
|
| 14 |
from free_claude_code.cli.process_registry import kill_all_best_effort
|
| 15 |
from free_claude_code.config.env_migrations import (
|
| 16 |
-
|
| 17 |
migrate_owned_env_files,
|
| 18 |
)
|
| 19 |
from free_claude_code.config.env_template import load_env_template
|
|
@@ -148,6 +148,6 @@ def _migrate_config_env_keys() -> tuple[Path, ...]:
|
|
| 148 |
"""Apply dotenv key migrations before Settings loads config."""
|
| 149 |
|
| 150 |
migrated = migrate_owned_env_files()
|
| 151 |
-
if warning :=
|
| 152 |
print(warning, file=sys.stderr)
|
| 153 |
return migrated
|
|
|
|
| 13 |
from free_claude_code.cli.launchers.common import preflight_proxy
|
| 14 |
from free_claude_code.cli.process_registry import kill_all_best_effort
|
| 15 |
from free_claude_code.config.env_migrations import (
|
| 16 |
+
explicit_env_file_migration_warning,
|
| 17 |
migrate_owned_env_files,
|
| 18 |
)
|
| 19 |
from free_claude_code.config.env_template import load_env_template
|
|
|
|
| 148 |
"""Apply dotenv key migrations before Settings loads config."""
|
| 149 |
|
| 150 |
migrated = migrate_owned_env_files()
|
| 151 |
+
if warning := explicit_env_file_migration_warning(os.environ):
|
| 152 |
print(warning, file=sys.stderr)
|
| 153 |
return migrated
|
|
@@ -4,6 +4,11 @@ from collections.abc import Iterable
|
|
| 4 |
from dataclasses import dataclass
|
| 5 |
from typing import Literal
|
| 6 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
from free_claude_code.config.settings import Settings
|
| 8 |
|
| 9 |
from .provider_manifest import provider_field_specs
|
|
@@ -13,7 +18,6 @@ FieldType = Literal[
|
|
| 13 |
"secret",
|
| 14 |
"number",
|
| 15 |
"boolean",
|
| 16 |
-
"tri_boolean",
|
| 17 |
"model",
|
| 18 |
"optional_model",
|
| 19 |
"select",
|
|
@@ -41,7 +45,7 @@ class ConfigFieldSpec:
|
|
| 41 |
field_type: FieldType = "text"
|
| 42 |
settings_attr: str | None = None
|
| 43 |
default: str = ""
|
| 44 |
-
options: tuple[str, ...] = ()
|
| 45 |
secret: bool = False
|
| 46 |
advanced: bool = False
|
| 47 |
restart_required: bool = False
|
|
@@ -49,6 +53,33 @@ class ConfigFieldSpec:
|
|
| 49 |
description: str = ""
|
| 50 |
|
| 51 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
SECTIONS: tuple[ConfigSectionSpec, ...] = (
|
| 53 |
ConfigSectionSpec(
|
| 54 |
"providers",
|
|
@@ -61,9 +92,9 @@ SECTIONS: tuple[ConfigSectionSpec, ...] = (
|
|
| 61 |
"Search discovered provider models or enter a provider/model slug.",
|
| 62 |
),
|
| 63 |
ConfigSectionSpec(
|
| 64 |
-
"
|
| 65 |
-
"
|
| 66 |
-
"
|
| 67 |
),
|
| 68 |
ConfigSectionSpec(
|
| 69 |
"runtime",
|
|
@@ -143,44 +174,53 @@ _NON_PROVIDER_FIELDS: tuple[ConfigFieldSpec, ...] = (
|
|
| 143 |
description="Select None to use the Default Model for Haiku requests.",
|
| 144 |
),
|
| 145 |
ConfigFieldSpec(
|
| 146 |
-
"
|
| 147 |
-
"
|
| 148 |
-
"
|
| 149 |
-
"
|
| 150 |
-
settings_attr="
|
| 151 |
-
default="
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 160 |
),
|
| 161 |
ConfigFieldSpec(
|
| 162 |
-
"
|
| 163 |
-
"Opus
|
| 164 |
-
"
|
| 165 |
-
"
|
| 166 |
-
settings_attr="
|
| 167 |
-
|
|
|
|
| 168 |
),
|
| 169 |
ConfigFieldSpec(
|
| 170 |
-
"
|
| 171 |
-
"Sonnet
|
| 172 |
-
"
|
| 173 |
-
"
|
| 174 |
-
settings_attr="
|
| 175 |
-
|
|
|
|
| 176 |
),
|
| 177 |
ConfigFieldSpec(
|
| 178 |
-
"
|
| 179 |
-
"Haiku
|
| 180 |
-
"
|
| 181 |
-
"
|
| 182 |
-
settings_attr="
|
| 183 |
-
|
|
|
|
| 184 |
),
|
| 185 |
ConfigFieldSpec(
|
| 186 |
"ANTHROPIC_AUTH_TOKEN",
|
|
|
|
| 4 |
from dataclasses import dataclass
|
| 5 |
from typing import Literal
|
| 6 |
|
| 7 |
+
from free_claude_code.config.reasoning import (
|
| 8 |
+
ROOT_REASONING_PREFERENCES,
|
| 9 |
+
ROUTE_REASONING_PREFERENCES,
|
| 10 |
+
ReasoningPreference,
|
| 11 |
+
)
|
| 12 |
from free_claude_code.config.settings import Settings
|
| 13 |
|
| 14 |
from .provider_manifest import provider_field_specs
|
|
|
|
| 18 |
"secret",
|
| 19 |
"number",
|
| 20 |
"boolean",
|
|
|
|
| 21 |
"model",
|
| 22 |
"optional_model",
|
| 23 |
"select",
|
|
|
|
| 45 |
field_type: FieldType = "text"
|
| 46 |
settings_attr: str | None = None
|
| 47 |
default: str = ""
|
| 48 |
+
options: tuple[str | ConfigOptionSpec, ...] = ()
|
| 49 |
secret: bool = False
|
| 50 |
advanced: bool = False
|
| 51 |
restart_required: bool = False
|
|
|
|
| 53 |
description: str = ""
|
| 54 |
|
| 55 |
|
| 56 |
+
@dataclass(frozen=True, slots=True)
|
| 57 |
+
class ConfigOptionSpec:
|
| 58 |
+
"""A persisted option value and its user-facing label."""
|
| 59 |
+
|
| 60 |
+
value: str
|
| 61 |
+
label: str
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
def _reasoning_options(
|
| 65 |
+
preferences: tuple[ReasoningPreference, ...],
|
| 66 |
+
) -> tuple[ConfigOptionSpec, ...]:
|
| 67 |
+
labels = {
|
| 68 |
+
ReasoningPreference.INHERIT: "Inherit",
|
| 69 |
+
ReasoningPreference.OFF: "Off",
|
| 70 |
+
ReasoningPreference.CLIENT: "From client",
|
| 71 |
+
ReasoningPreference.LOW: "Low",
|
| 72 |
+
ReasoningPreference.MEDIUM: "Medium",
|
| 73 |
+
ReasoningPreference.HIGH: "High",
|
| 74 |
+
ReasoningPreference.XHIGH: "X-High",
|
| 75 |
+
ReasoningPreference.MAX: "Max",
|
| 76 |
+
}
|
| 77 |
+
return tuple(
|
| 78 |
+
ConfigOptionSpec(preference.value, labels[preference])
|
| 79 |
+
for preference in preferences
|
| 80 |
+
)
|
| 81 |
+
|
| 82 |
+
|
| 83 |
SECTIONS: tuple[ConfigSectionSpec, ...] = (
|
| 84 |
ConfigSectionSpec(
|
| 85 |
"providers",
|
|
|
|
| 92 |
"Search discovered provider models or enter a provider/model slug.",
|
| 93 |
),
|
| 94 |
ConfigSectionSpec(
|
| 95 |
+
"reasoning",
|
| 96 |
+
"Reasoning",
|
| 97 |
+
"Client reasoning policy and route-specific overrides.",
|
| 98 |
),
|
| 99 |
ConfigSectionSpec(
|
| 100 |
"runtime",
|
|
|
|
| 174 |
description="Select None to use the Default Model for Haiku requests.",
|
| 175 |
),
|
| 176 |
ConfigFieldSpec(
|
| 177 |
+
"REASONING_POLICY",
|
| 178 |
+
"Reasoning Policy",
|
| 179 |
+
"reasoning",
|
| 180 |
+
"select",
|
| 181 |
+
settings_attr="reasoning_policy",
|
| 182 |
+
default="client",
|
| 183 |
+
options=_reasoning_options(ROOT_REASONING_PREFERENCES),
|
| 184 |
+
description=(
|
| 185 |
+
"From client preserves CLI effort. Providers translate only the controls "
|
| 186 |
+
"their API supports."
|
| 187 |
+
),
|
| 188 |
+
),
|
| 189 |
+
ConfigFieldSpec(
|
| 190 |
+
"REASONING_FABLE",
|
| 191 |
+
"Fable Reasoning",
|
| 192 |
+
"reasoning",
|
| 193 |
+
"select",
|
| 194 |
+
settings_attr="reasoning_fable",
|
| 195 |
+
default="inherit",
|
| 196 |
+
options=_reasoning_options(ROUTE_REASONING_PREFERENCES),
|
| 197 |
),
|
| 198 |
ConfigFieldSpec(
|
| 199 |
+
"REASONING_OPUS",
|
| 200 |
+
"Opus Reasoning",
|
| 201 |
+
"reasoning",
|
| 202 |
+
"select",
|
| 203 |
+
settings_attr="reasoning_opus",
|
| 204 |
+
default="inherit",
|
| 205 |
+
options=_reasoning_options(ROUTE_REASONING_PREFERENCES),
|
| 206 |
),
|
| 207 |
ConfigFieldSpec(
|
| 208 |
+
"REASONING_SONNET",
|
| 209 |
+
"Sonnet Reasoning",
|
| 210 |
+
"reasoning",
|
| 211 |
+
"select",
|
| 212 |
+
settings_attr="reasoning_sonnet",
|
| 213 |
+
default="inherit",
|
| 214 |
+
options=_reasoning_options(ROUTE_REASONING_PREFERENCES),
|
| 215 |
),
|
| 216 |
ConfigFieldSpec(
|
| 217 |
+
"REASONING_HAIKU",
|
| 218 |
+
"Haiku Reasoning",
|
| 219 |
+
"reasoning",
|
| 220 |
+
"select",
|
| 221 |
+
settings_attr="reasoning_haiku",
|
| 222 |
+
default="inherit",
|
| 223 |
+
options=_reasoning_options(ROUTE_REASONING_PREFERENCES),
|
| 224 |
),
|
| 225 |
ConfigFieldSpec(
|
| 226 |
"ANTHROPIC_AUTH_TOKEN",
|
|
@@ -5,7 +5,13 @@ from typing import Any
|
|
| 5 |
|
| 6 |
from free_claude_code.config.paths import managed_env_path
|
| 7 |
|
| 8 |
-
from .manifest import
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
from .sources import (
|
| 10 |
configured_env_files,
|
| 11 |
dotenv_values_from_file,
|
|
@@ -88,7 +94,14 @@ def load_config_response() -> dict[str, Any]:
|
|
| 88 |
"advanced": field.advanced,
|
| 89 |
"restart_required": field.restart_required,
|
| 90 |
"session_sensitive": field.session_sensitive,
|
| 91 |
-
"options":
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
"description": field.description,
|
| 93 |
}
|
| 94 |
)
|
|
|
|
| 5 |
|
| 6 |
from free_claude_code.config.paths import managed_env_path
|
| 7 |
|
| 8 |
+
from .manifest import (
|
| 9 |
+
FIELD_BY_KEY,
|
| 10 |
+
FIELDS,
|
| 11 |
+
SECTIONS,
|
| 12 |
+
ConfigFieldSpec,
|
| 13 |
+
ConfigOptionSpec,
|
| 14 |
+
)
|
| 15 |
from .sources import (
|
| 16 |
configured_env_files,
|
| 17 |
dotenv_values_from_file,
|
|
|
|
| 94 |
"advanced": field.advanced,
|
| 95 |
"restart_required": field.restart_required,
|
| 96 |
"session_sensitive": field.session_sensitive,
|
| 97 |
+
"options": [
|
| 98 |
+
(
|
| 99 |
+
{"value": option.value, "label": option.label}
|
| 100 |
+
if isinstance(option, ConfigOptionSpec)
|
| 101 |
+
else {"value": option, "label": option}
|
| 102 |
+
)
|
| 103 |
+
for option in field.options
|
| 104 |
+
],
|
| 105 |
"description": field.description,
|
| 106 |
}
|
| 107 |
)
|
|
@@ -22,6 +22,7 @@ class EnvKeyMigration:
|
|
| 22 |
|
| 23 |
old_key: str
|
| 24 |
new_key: str
|
|
|
|
| 25 |
|
| 26 |
|
| 27 |
HUGGINGFACE_TOKEN_MIGRATION = EnvKeyMigration(
|
|
@@ -29,32 +30,67 @@ HUGGINGFACE_TOKEN_MIGRATION = EnvKeyMigration(
|
|
| 29 |
new_key=HUGGINGFACE_API_KEY_ENV,
|
| 30 |
)
|
| 31 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
def migrate_owned_env_files() -> tuple[Path, ...]:
|
| 34 |
"""Apply key migrations to repo and managed dotenv files."""
|
| 35 |
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
|
| 43 |
-
def
|
| 44 |
env: Mapping[str, str] | None = None,
|
| 45 |
) -> str | None:
|
| 46 |
-
"""Return a warning when an explicit env file
|
| 47 |
|
| 48 |
path = explicit_env_path(env)
|
| 49 |
if path is None or not path.is_file():
|
| 50 |
return None
|
| 51 |
text = path.read_text(encoding="utf-8")
|
| 52 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
return None
|
|
|
|
|
|
|
|
|
|
| 54 |
return (
|
| 55 |
-
f"{
|
| 56 |
-
|
| 57 |
-
"rewritten automatically."
|
| 58 |
)
|
| 59 |
|
| 60 |
|
|
@@ -86,9 +122,12 @@ def migrate_env_key_in_text(
|
|
| 86 |
match = _DOTENV_ASSIGNMENT_RE.match(line)
|
| 87 |
if match is None or match.group("key") != migration.old_key:
|
| 88 |
continue
|
|
|
|
|
|
|
|
|
|
| 89 |
lines[index] = (
|
| 90 |
f"{match.group('prefix')}{migration.new_key}{match.group('suffix')}"
|
| 91 |
-
f"{
|
| 92 |
)
|
| 93 |
changed = True
|
| 94 |
if not changed:
|
|
@@ -114,6 +153,20 @@ def _defines_key(text: str, key: str) -> bool:
|
|
| 114 |
return False
|
| 115 |
|
| 116 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 117 |
def _unique_paths(paths: tuple[Path, ...]) -> tuple[Path, ...]:
|
| 118 |
seen: set[Path] = set()
|
| 119 |
unique: list[Path] = []
|
|
|
|
| 22 |
|
| 23 |
old_key: str
|
| 24 |
new_key: str
|
| 25 |
+
value_map: tuple[tuple[str, str], ...] = ()
|
| 26 |
|
| 27 |
|
| 28 |
HUGGINGFACE_TOKEN_MIGRATION = EnvKeyMigration(
|
|
|
|
| 30 |
new_key=HUGGINGFACE_API_KEY_ENV,
|
| 31 |
)
|
| 32 |
|
| 33 |
+
_LEGACY_TRUE_VALUES = ("1", "true", "t", "on", "yes", "y")
|
| 34 |
+
_LEGACY_FALSE_VALUES = ("0", "false", "f", "off", "no", "n")
|
| 35 |
+
_LEGACY_REASONING_BOOLEAN_MAP = (
|
| 36 |
+
*((value, "client") for value in _LEGACY_TRUE_VALUES),
|
| 37 |
+
*((value, "off") for value in _LEGACY_FALSE_VALUES),
|
| 38 |
+
)
|
| 39 |
+
|
| 40 |
+
REASONING_MIGRATIONS = (
|
| 41 |
+
EnvKeyMigration(
|
| 42 |
+
"ENABLE_MODEL_THINKING",
|
| 43 |
+
"REASONING_POLICY",
|
| 44 |
+
_LEGACY_REASONING_BOOLEAN_MAP,
|
| 45 |
+
),
|
| 46 |
+
*(
|
| 47 |
+
EnvKeyMigration(
|
| 48 |
+
f"ENABLE_{route}_THINKING",
|
| 49 |
+
f"REASONING_{route}",
|
| 50 |
+
(("", "inherit"), *_LEGACY_REASONING_BOOLEAN_MAP),
|
| 51 |
+
)
|
| 52 |
+
for route in ("FABLE", "OPUS", "SONNET", "HAIKU")
|
| 53 |
+
),
|
| 54 |
+
)
|
| 55 |
+
|
| 56 |
+
ENV_MIGRATIONS = (HUGGINGFACE_TOKEN_MIGRATION, *REASONING_MIGRATIONS)
|
| 57 |
+
|
| 58 |
|
| 59 |
def migrate_owned_env_files() -> tuple[Path, ...]:
|
| 60 |
"""Apply key migrations to repo and managed dotenv files."""
|
| 61 |
|
| 62 |
+
changed_paths: list[Path] = []
|
| 63 |
+
for path in _unique_paths((repo_env_path(), managed_env_path())):
|
| 64 |
+
changed = False
|
| 65 |
+
for migration in ENV_MIGRATIONS:
|
| 66 |
+
changed = migrate_env_key_in_file(path, migration) or changed
|
| 67 |
+
if changed:
|
| 68 |
+
changed_paths.append(path.resolve())
|
| 69 |
+
return tuple(changed_paths)
|
| 70 |
|
| 71 |
|
| 72 |
+
def explicit_env_file_migration_warning(
|
| 73 |
env: Mapping[str, str] | None = None,
|
| 74 |
) -> str | None:
|
| 75 |
+
"""Return a warning when an explicit env file uses a retired setting."""
|
| 76 |
|
| 77 |
path = explicit_env_path(env)
|
| 78 |
if path is None or not path.is_file():
|
| 79 |
return None
|
| 80 |
text = path.read_text(encoding="utf-8")
|
| 81 |
+
pending = tuple(
|
| 82 |
+
migration
|
| 83 |
+
for migration in ENV_MIGRATIONS
|
| 84 |
+
if env_text_needs_migration(text, migration)
|
| 85 |
+
)
|
| 86 |
+
if not pending:
|
| 87 |
return None
|
| 88 |
+
renames = ", ".join(
|
| 89 |
+
f"{migration.old_key} to {migration.new_key}" for migration in pending
|
| 90 |
+
)
|
| 91 |
return (
|
| 92 |
+
f"Explicit FCC_ENV_FILE {path} uses retired settings. Rename {renames}; "
|
| 93 |
+
"explicit env files are not rewritten automatically."
|
|
|
|
| 94 |
)
|
| 95 |
|
| 96 |
|
|
|
|
| 122 |
match = _DOTENV_ASSIGNMENT_RE.match(line)
|
| 123 |
if match is None or match.group("key") != migration.old_key:
|
| 124 |
continue
|
| 125 |
+
remainder = line[match.end() :]
|
| 126 |
+
if migration.value_map:
|
| 127 |
+
remainder = _mapped_value(remainder, migration.value_map)
|
| 128 |
lines[index] = (
|
| 129 |
f"{match.group('prefix')}{migration.new_key}{match.group('suffix')}"
|
| 130 |
+
f"{remainder}"
|
| 131 |
)
|
| 132 |
changed = True
|
| 133 |
if not changed:
|
|
|
|
| 153 |
return False
|
| 154 |
|
| 155 |
|
| 156 |
+
def _mapped_value(value: str, mapping: tuple[tuple[str, str], ...]) -> str:
|
| 157 |
+
"""Map a simple dotenv value while preserving comments and line endings."""
|
| 158 |
+
|
| 159 |
+
line = value.rstrip("\r\n")
|
| 160 |
+
newline = value[len(line) :]
|
| 161 |
+
raw_value, separator, comment = line.partition("#")
|
| 162 |
+
normalized = raw_value.strip().strip("'\"").lower()
|
| 163 |
+
replacement = dict(mapping).get(normalized)
|
| 164 |
+
if replacement is None:
|
| 165 |
+
return value
|
| 166 |
+
suffix = f" #{comment}" if separator else ""
|
| 167 |
+
return f"{replacement}{suffix}{newline}"
|
| 168 |
+
|
| 169 |
+
|
| 170 |
def _unique_paths(paths: tuple[Path, ...]) -> tuple[Path, ...]:
|
| 171 |
seen: set[Path] = set()
|
| 172 |
unique: list[Path] = []
|
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""User-configurable reasoning policy values."""
|
| 2 |
+
|
| 3 |
+
from enum import StrEnum
|
| 4 |
+
|
| 5 |
+
|
| 6 |
+
class ReasoningPreference(StrEnum):
|
| 7 |
+
"""Configuration choice applied before provider translation."""
|
| 8 |
+
|
| 9 |
+
INHERIT = "inherit"
|
| 10 |
+
OFF = "off"
|
| 11 |
+
CLIENT = "client"
|
| 12 |
+
LOW = "low"
|
| 13 |
+
MEDIUM = "medium"
|
| 14 |
+
HIGH = "high"
|
| 15 |
+
XHIGH = "xhigh"
|
| 16 |
+
MAX = "max"
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
ROOT_REASONING_PREFERENCES = tuple(
|
| 20 |
+
preference
|
| 21 |
+
for preference in ReasoningPreference
|
| 22 |
+
if preference is not ReasoningPreference.INHERIT
|
| 23 |
+
)
|
| 24 |
+
ROUTE_REASONING_PREFERENCES = tuple(ReasoningPreference)
|
|
@@ -14,6 +14,7 @@ from .env_files import (
|
|
| 14 |
)
|
| 15 |
from .nim import NimSettings
|
| 16 |
from .provider_catalog import SUPPORTED_PROVIDER_IDS
|
|
|
|
| 17 |
|
| 18 |
|
| 19 |
class Settings(BaseSettings):
|
|
@@ -167,20 +168,25 @@ class Settings(BaseSettings):
|
|
| 167 |
provider_max_concurrency: int = Field(
|
| 168 |
default=5, validation_alias="PROVIDER_MAX_CONCURRENCY"
|
| 169 |
)
|
| 170 |
-
|
| 171 |
-
default=
|
|
|
|
| 172 |
)
|
| 173 |
-
|
| 174 |
-
default=
|
|
|
|
| 175 |
)
|
| 176 |
-
|
| 177 |
-
default=
|
|
|
|
| 178 |
)
|
| 179 |
-
|
| 180 |
-
default=
|
|
|
|
| 181 |
)
|
| 182 |
-
|
| 183 |
-
default=
|
|
|
|
| 184 |
)
|
| 185 |
|
| 186 |
# ==================== HTTP Client Timeouts ====================
|
|
@@ -301,10 +307,6 @@ class Settings(BaseSettings):
|
|
| 301 |
"model_opus",
|
| 302 |
"model_sonnet",
|
| 303 |
"model_haiku",
|
| 304 |
-
"enable_fable_thinking",
|
| 305 |
-
"enable_opus_thinking",
|
| 306 |
-
"enable_sonnet_thinking",
|
| 307 |
-
"enable_haiku_thinking",
|
| 308 |
mode="before",
|
| 309 |
)
|
| 310 |
@classmethod
|
|
@@ -329,6 +331,15 @@ class Settings(BaseSettings):
|
|
| 329 |
raise ValueError(f"LOG_LEVEL must be one of {sorted(valid)}, got {v!r}")
|
| 330 |
return upper
|
| 331 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 332 |
@field_validator("whisper_device")
|
| 333 |
@classmethod
|
| 334 |
def validate_whisper_device(cls, v: str) -> str:
|
|
|
|
| 14 |
)
|
| 15 |
from .nim import NimSettings
|
| 16 |
from .provider_catalog import SUPPORTED_PROVIDER_IDS
|
| 17 |
+
from .reasoning import ReasoningPreference
|
| 18 |
|
| 19 |
|
| 20 |
class Settings(BaseSettings):
|
|
|
|
| 168 |
provider_max_concurrency: int = Field(
|
| 169 |
default=5, validation_alias="PROVIDER_MAX_CONCURRENCY"
|
| 170 |
)
|
| 171 |
+
reasoning_policy: ReasoningPreference = Field(
|
| 172 |
+
default=ReasoningPreference.CLIENT,
|
| 173 |
+
validation_alias="REASONING_POLICY",
|
| 174 |
)
|
| 175 |
+
reasoning_fable: ReasoningPreference = Field(
|
| 176 |
+
default=ReasoningPreference.INHERIT,
|
| 177 |
+
validation_alias="REASONING_FABLE",
|
| 178 |
)
|
| 179 |
+
reasoning_opus: ReasoningPreference = Field(
|
| 180 |
+
default=ReasoningPreference.INHERIT,
|
| 181 |
+
validation_alias="REASONING_OPUS",
|
| 182 |
)
|
| 183 |
+
reasoning_sonnet: ReasoningPreference = Field(
|
| 184 |
+
default=ReasoningPreference.INHERIT,
|
| 185 |
+
validation_alias="REASONING_SONNET",
|
| 186 |
)
|
| 187 |
+
reasoning_haiku: ReasoningPreference = Field(
|
| 188 |
+
default=ReasoningPreference.INHERIT,
|
| 189 |
+
validation_alias="REASONING_HAIKU",
|
| 190 |
)
|
| 191 |
|
| 192 |
# ==================== HTTP Client Timeouts ====================
|
|
|
|
| 307 |
"model_opus",
|
| 308 |
"model_sonnet",
|
| 309 |
"model_haiku",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 310 |
mode="before",
|
| 311 |
)
|
| 312 |
@classmethod
|
|
|
|
| 331 |
raise ValueError(f"LOG_LEVEL must be one of {sorted(valid)}, got {v!r}")
|
| 332 |
return upper
|
| 333 |
|
| 334 |
+
@field_validator("reasoning_policy")
|
| 335 |
+
@classmethod
|
| 336 |
+
def validate_root_reasoning_policy(
|
| 337 |
+
cls, value: ReasoningPreference
|
| 338 |
+
) -> ReasoningPreference:
|
| 339 |
+
if value is ReasoningPreference.INHERIT:
|
| 340 |
+
raise ValueError("REASONING_POLICY cannot inherit")
|
| 341 |
+
return value
|
| 342 |
+
|
| 343 |
@field_validator("whisper_device")
|
| 344 |
@classmethod
|
| 345 |
def validate_whisper_device(cls, v: str) -> str:
|
|
@@ -1 +1,15 @@
|
|
| 1 |
"""Neutral shared application core."""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
"""Neutral shared application core."""
|
| 2 |
+
|
| 3 |
+
from .reasoning import (
|
| 4 |
+
DEFAULT_REASONING_POLICY,
|
| 5 |
+
ReasoningControl,
|
| 6 |
+
ReasoningEffort,
|
| 7 |
+
ReasoningPolicy,
|
| 8 |
+
)
|
| 9 |
+
|
| 10 |
+
__all__ = [
|
| 11 |
+
"DEFAULT_REASONING_POLICY",
|
| 12 |
+
"ReasoningControl",
|
| 13 |
+
"ReasoningEffort",
|
| 14 |
+
"ReasoningPolicy",
|
| 15 |
+
]
|
|
@@ -28,7 +28,6 @@ def anthropic_request_snapshot(
|
|
| 28 |
"stop_sequences",
|
| 29 |
"metadata",
|
| 30 |
"stream",
|
| 31 |
-
"thinking_enabled",
|
| 32 |
)
|
| 33 |
if key in data and data[key] is not None
|
| 34 |
}
|
|
|
|
| 28 |
"stop_sequences",
|
| 29 |
"metadata",
|
| 30 |
"stream",
|
|
|
|
| 31 |
)
|
| 32 |
if key in data and data[key] is not None
|
| 33 |
}
|
|
@@ -14,7 +14,7 @@ NO_THINKING_GATEWAY_MODEL_ID_PREFIX = "claude-3-freecc-no-thinking"
|
|
| 14 |
class DecodedGatewayModelId:
|
| 15 |
provider_id: str
|
| 16 |
provider_model: str
|
| 17 |
-
|
| 18 |
|
| 19 |
|
| 20 |
def gateway_model_id(provider_model_ref: str) -> str:
|
|
@@ -33,11 +33,10 @@ def decode_gateway_model_id(model_name: str) -> DecodedGatewayModelId | None:
|
|
| 33 |
if not separator:
|
| 34 |
return None
|
| 35 |
|
| 36 |
-
force_thinking_enabled: bool | None
|
| 37 |
if prefix == GATEWAY_MODEL_ID_PREFIX:
|
| 38 |
-
|
| 39 |
elif prefix == NO_THINKING_GATEWAY_MODEL_ID_PREFIX:
|
| 40 |
-
|
| 41 |
else:
|
| 42 |
return None
|
| 43 |
|
|
@@ -48,5 +47,5 @@ def decode_gateway_model_id(model_name: str) -> DecodedGatewayModelId | None:
|
|
| 48 |
return DecodedGatewayModelId(
|
| 49 |
provider_id=provider_id,
|
| 50 |
provider_model=provider_model,
|
| 51 |
-
|
| 52 |
)
|
|
|
|
| 14 |
class DecodedGatewayModelId:
|
| 15 |
provider_id: str
|
| 16 |
provider_model: str
|
| 17 |
+
force_reasoning_off: bool = False
|
| 18 |
|
| 19 |
|
| 20 |
def gateway_model_id(provider_model_ref: str) -> str:
|
|
|
|
| 33 |
if not separator:
|
| 34 |
return None
|
| 35 |
|
|
|
|
| 36 |
if prefix == GATEWAY_MODEL_ID_PREFIX:
|
| 37 |
+
force_reasoning_off = False
|
| 38 |
elif prefix == NO_THINKING_GATEWAY_MODEL_ID_PREFIX:
|
| 39 |
+
force_reasoning_off = True
|
| 40 |
else:
|
| 41 |
return None
|
| 42 |
|
|
|
|
| 47 |
return DecodedGatewayModelId(
|
| 48 |
provider_id=provider_id,
|
| 49 |
provider_model=provider_model,
|
| 50 |
+
force_reasoning_off=force_reasoning_off,
|
| 51 |
)
|
|
@@ -10,7 +10,7 @@ from .models import OpenAIResponsesRequest
|
|
| 10 |
from .reasoning import (
|
| 11 |
combine_reasoning,
|
| 12 |
reasoning_text_from_item,
|
| 13 |
-
|
| 14 |
)
|
| 15 |
from .tools import (
|
| 16 |
call_id_from_item,
|
|
@@ -65,8 +65,8 @@ def convert_request_to_anthropic_payload(
|
|
| 65 |
if request.metadata is not None:
|
| 66 |
payload["metadata"] = request.metadata
|
| 67 |
|
| 68 |
-
if
|
| 69 |
-
payload["
|
| 70 |
|
| 71 |
raw_tool_choice = request.tool_choice
|
| 72 |
tools = convert_tools(request.tools)
|
|
|
|
| 10 |
from .reasoning import (
|
| 11 |
combine_reasoning,
|
| 12 |
reasoning_text_from_item,
|
| 13 |
+
responses_reasoning_to_output_config,
|
| 14 |
)
|
| 15 |
from .tools import (
|
| 16 |
call_id_from_item,
|
|
|
|
| 65 |
if request.metadata is not None:
|
| 66 |
payload["metadata"] = request.metadata
|
| 67 |
|
| 68 |
+
if output_config := responses_reasoning_to_output_config(request.reasoning):
|
| 69 |
+
payload["output_config"] = output_config
|
| 70 |
|
| 71 |
raw_tool_choice = request.tool_choice
|
| 72 |
tools = convert_tools(request.tools)
|
|
@@ -32,13 +32,13 @@ def combine_reasoning(existing: str | None, addition: str | None) -> str | None:
|
|
| 32 |
return f"{existing}\n{addition}"
|
| 33 |
|
| 34 |
|
| 35 |
-
def
|
|
|
|
| 36 |
if not isinstance(value, Mapping):
|
| 37 |
return None
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
return {"type": "enabled", "enabled": True}
|
| 42 |
return None
|
| 43 |
|
| 44 |
|
|
|
|
| 32 |
return f"{existing}\n{addition}"
|
| 33 |
|
| 34 |
|
| 35 |
+
def responses_reasoning_to_output_config(value: Any) -> dict[str, Any] | None:
|
| 36 |
+
"""Preserve the client's named effort for application-level resolution."""
|
| 37 |
if not isinstance(value, Mapping):
|
| 38 |
return None
|
| 39 |
+
effort = value.get("effort")
|
| 40 |
+
if isinstance(effort, str) and effort.strip():
|
| 41 |
+
return {"effort": effort.strip().lower()}
|
|
|
|
| 42 |
return None
|
| 43 |
|
| 44 |
|
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Provider-neutral reasoning intent."""
|
| 2 |
+
|
| 3 |
+
from dataclasses import dataclass
|
| 4 |
+
from enum import StrEnum
|
| 5 |
+
|
| 6 |
+
|
| 7 |
+
class ReasoningControl(StrEnum):
|
| 8 |
+
"""Whether a request explicitly controls reasoning computation."""
|
| 9 |
+
|
| 10 |
+
DEFAULT = "default"
|
| 11 |
+
OFF = "off"
|
| 12 |
+
ON = "on"
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
class ReasoningEffort(StrEnum):
|
| 16 |
+
"""Named reasoning effort understood at the FCC application boundary."""
|
| 17 |
+
|
| 18 |
+
MINIMAL = "minimal"
|
| 19 |
+
LOW = "low"
|
| 20 |
+
MEDIUM = "medium"
|
| 21 |
+
HIGH = "high"
|
| 22 |
+
XHIGH = "xhigh"
|
| 23 |
+
MAX = "max"
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
@dataclass(frozen=True, slots=True)
|
| 27 |
+
class ReasoningPolicy:
|
| 28 |
+
"""Resolved client and configuration intent passed to one provider.
|
| 29 |
+
|
| 30 |
+
``control`` and ``effort`` remain independent because clients may set an
|
| 31 |
+
overall effort while separately disabling extended thinking. Providers
|
| 32 |
+
translate the representable subset without changing the original intent.
|
| 33 |
+
"""
|
| 34 |
+
|
| 35 |
+
control: ReasoningControl = ReasoningControl.DEFAULT
|
| 36 |
+
effort: ReasoningEffort | None = None
|
| 37 |
+
budget_tokens: int | None = None
|
| 38 |
+
|
| 39 |
+
def __post_init__(self) -> None:
|
| 40 |
+
if self.budget_tokens is not None and (
|
| 41 |
+
not isinstance(self.budget_tokens, int)
|
| 42 |
+
or isinstance(self.budget_tokens, bool)
|
| 43 |
+
or self.budget_tokens <= 0
|
| 44 |
+
):
|
| 45 |
+
raise ValueError("Reasoning budget must be a positive integer.")
|
| 46 |
+
if self.budget_tokens is not None and self.control is not ReasoningControl.ON:
|
| 47 |
+
raise ValueError("A reasoning budget requires reasoning control to be on.")
|
| 48 |
+
|
| 49 |
+
@classmethod
|
| 50 |
+
def provider_default(cls) -> ReasoningPolicy:
|
| 51 |
+
"""Leave reasoning computation to the provider."""
|
| 52 |
+
|
| 53 |
+
return cls()
|
| 54 |
+
|
| 55 |
+
@classmethod
|
| 56 |
+
def off(cls) -> ReasoningPolicy:
|
| 57 |
+
"""Explicitly disable reasoning computation and output."""
|
| 58 |
+
|
| 59 |
+
return cls(control=ReasoningControl.OFF)
|
| 60 |
+
|
| 61 |
+
@classmethod
|
| 62 |
+
def on(
|
| 63 |
+
cls,
|
| 64 |
+
*,
|
| 65 |
+
effort: ReasoningEffort | None = None,
|
| 66 |
+
budget_tokens: int | None = None,
|
| 67 |
+
) -> ReasoningPolicy:
|
| 68 |
+
"""Explicitly enable reasoning with optional client controls."""
|
| 69 |
+
|
| 70 |
+
return cls(
|
| 71 |
+
control=ReasoningControl.ON,
|
| 72 |
+
effort=effort,
|
| 73 |
+
budget_tokens=budget_tokens,
|
| 74 |
+
)
|
| 75 |
+
|
| 76 |
+
@property
|
| 77 |
+
def output_enabled(self) -> bool:
|
| 78 |
+
"""Return whether provider reasoning may be exposed to the client."""
|
| 79 |
+
|
| 80 |
+
return self.control is not ReasoningControl.OFF
|
| 81 |
+
|
| 82 |
+
@property
|
| 83 |
+
def requests_reasoning(self) -> bool:
|
| 84 |
+
"""Return whether the request explicitly asks the provider to reason."""
|
| 85 |
+
|
| 86 |
+
return self.control is not ReasoningControl.OFF and (
|
| 87 |
+
self.control is ReasoningControl.ON
|
| 88 |
+
or self.effort is not None
|
| 89 |
+
or self.budget_tokens is not None
|
| 90 |
+
)
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
DEFAULT_REASONING_POLICY = ReasoningPolicy.provider_default()
|
|
@@ -13,6 +13,7 @@ from free_claude_code.core.diagnostics import (
|
|
| 13 |
exception_cause_types,
|
| 14 |
redacted_exception_traceback,
|
| 15 |
)
|
|
|
|
| 16 |
from free_claude_code.core.trace import trace_event
|
| 17 |
from free_claude_code.providers.model_listing import model_infos_from_ids
|
| 18 |
|
|
@@ -33,7 +34,6 @@ class ProviderConfig:
|
|
| 33 |
http_read_timeout: float = 300.0
|
| 34 |
http_write_timeout: float = 10.0
|
| 35 |
http_connect_timeout: float = HTTP_CONNECT_TIMEOUT_DEFAULT
|
| 36 |
-
enable_thinking: bool = True
|
| 37 |
proxy: str = ""
|
| 38 |
log_raw_sse_events: bool = False
|
| 39 |
log_api_error_tracebacks: bool = False
|
|
@@ -45,27 +45,12 @@ class BaseProvider(ABC):
|
|
| 45 |
def __init__(self, config: ProviderConfig):
|
| 46 |
self._config = config
|
| 47 |
|
| 48 |
-
def _is_thinking_enabled(
|
| 49 |
-
self, request: MessagesRequest, thinking_enabled: bool | None = None
|
| 50 |
-
) -> bool:
|
| 51 |
-
"""Return whether thinking should be enabled for this request."""
|
| 52 |
-
thinking = request.thinking
|
| 53 |
-
config_enabled = (
|
| 54 |
-
self._config.enable_thinking
|
| 55 |
-
if thinking_enabled is None
|
| 56 |
-
else thinking_enabled
|
| 57 |
-
)
|
| 58 |
-
request_enabled = True
|
| 59 |
-
if thinking is not None:
|
| 60 |
-
if "enabled" in thinking.model_fields_set and thinking.enabled is not None:
|
| 61 |
-
request_enabled = thinking.enabled
|
| 62 |
-
if thinking.type == "disabled":
|
| 63 |
-
request_enabled = False
|
| 64 |
-
return config_enabled and request_enabled
|
| 65 |
-
|
| 66 |
@abstractmethod
|
| 67 |
def preflight_stream(
|
| 68 |
-
self,
|
|
|
|
|
|
|
|
|
|
| 69 |
) -> None:
|
| 70 |
"""Validate the upstream request before opening an SSE stream."""
|
| 71 |
|
|
@@ -131,6 +116,6 @@ class BaseProvider(ABC):
|
|
| 131 |
input_tokens: int = 0,
|
| 132 |
*,
|
| 133 |
request_id: str | None = None,
|
| 134 |
-
|
| 135 |
) -> AsyncIterator[str]:
|
| 136 |
"""Stream response in Anthropic SSE format."""
|
|
|
|
| 13 |
exception_cause_types,
|
| 14 |
redacted_exception_traceback,
|
| 15 |
)
|
| 16 |
+
from free_claude_code.core.reasoning import DEFAULT_REASONING_POLICY, ReasoningPolicy
|
| 17 |
from free_claude_code.core.trace import trace_event
|
| 18 |
from free_claude_code.providers.model_listing import model_infos_from_ids
|
| 19 |
|
|
|
|
| 34 |
http_read_timeout: float = 300.0
|
| 35 |
http_write_timeout: float = 10.0
|
| 36 |
http_connect_timeout: float = HTTP_CONNECT_TIMEOUT_DEFAULT
|
|
|
|
| 37 |
proxy: str = ""
|
| 38 |
log_raw_sse_events: bool = False
|
| 39 |
log_api_error_tracebacks: bool = False
|
|
|
|
| 45 |
def __init__(self, config: ProviderConfig):
|
| 46 |
self._config = config
|
| 47 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
@abstractmethod
|
| 49 |
def preflight_stream(
|
| 50 |
+
self,
|
| 51 |
+
request: MessagesRequest,
|
| 52 |
+
*,
|
| 53 |
+
reasoning: ReasoningPolicy = DEFAULT_REASONING_POLICY,
|
| 54 |
) -> None:
|
| 55 |
"""Validate the upstream request before opening an SSE stream."""
|
| 56 |
|
|
|
|
| 116 |
input_tokens: int = 0,
|
| 117 |
*,
|
| 118 |
request_id: str | None = None,
|
| 119 |
+
reasoning: ReasoningPolicy = DEFAULT_REASONING_POLICY,
|
| 120 |
) -> AsyncIterator[str]:
|
| 121 |
"""Stream response in Anthropic SSE format."""
|
|
@@ -10,7 +10,7 @@ import httpx
|
|
| 10 |
from free_claude_code.application.errors import ApplicationUnavailableError
|
| 11 |
from free_claude_code.application.model_metadata import ProviderModelInfo
|
| 12 |
from free_claude_code.config.provider_catalog import CLOUDFLARE_AI_REST_ROOT
|
| 13 |
-
from free_claude_code.core.anthropic
|
| 14 |
from free_claude_code.providers.base import ProviderConfig
|
| 15 |
from free_claude_code.providers.http import maybe_await_aclose
|
| 16 |
from free_claude_code.providers.model_listing import (
|
|
@@ -19,19 +19,22 @@ from free_claude_code.providers.model_listing import (
|
|
| 19 |
model_infos_from_ids,
|
| 20 |
)
|
| 21 |
from free_claude_code.providers.openai_chat import (
|
|
|
|
| 22 |
OpenAIChatProfile,
|
| 23 |
OpenAIChatProvider,
|
| 24 |
OpenAIChatRequestPolicy,
|
| 25 |
-
|
| 26 |
)
|
| 27 |
from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
| 28 |
|
| 29 |
_REQUEST_POLICY = OpenAIChatRequestPolicy(
|
| 30 |
provider_name="CLOUDFLARE",
|
|
|
|
| 31 |
include_extra_body=True,
|
|
|
|
| 32 |
max_tokens_field="max_completion_tokens",
|
| 33 |
)
|
| 34 |
-
_PROFILE = OpenAIChatProfile(_REQUEST_POLICY)
|
| 35 |
|
| 36 |
|
| 37 |
def cloudflare_ai_base_url(api_root: str | None, account_id: str) -> str:
|
|
@@ -118,22 +121,12 @@ class CloudflareProvider(OpenAIChatProvider):
|
|
| 118 |
finally:
|
| 119 |
await maybe_await_aclose(response)
|
| 120 |
|
| 121 |
-
def _build_request_body(
|
| 122 |
-
self, request: MessagesRequest, thinking_enabled: bool | None = None
|
| 123 |
-
) -> dict:
|
| 124 |
-
return build_openai_chat_request_body(
|
| 125 |
-
request,
|
| 126 |
-
thinking_enabled=self._is_thinking_enabled(request, thinking_enabled),
|
| 127 |
-
policy=_REQUEST_POLICY,
|
| 128 |
-
postprocessors=(_apply_cloudflare_request_quirks,),
|
| 129 |
-
)
|
| 130 |
-
|
| 131 |
def _handle_extra_reasoning(
|
| 132 |
-
self, delta: Any, ledger: Any, *,
|
| 133 |
) -> Iterator[str]:
|
| 134 |
"""Map Cloudflare's ``reasoning`` delta field to Anthropic thinking."""
|
| 135 |
reasoning = _cloudflare_reasoning(delta)
|
| 136 |
-
if not
|
| 137 |
return
|
| 138 |
yield from ledger.ensure_thinking_block()
|
| 139 |
yield ledger.emit_thinking_delta(reasoning)
|
|
@@ -142,18 +135,6 @@ class CloudflareProvider(OpenAIChatProvider):
|
|
| 142 |
return {"Authorization": f"Bearer {self._api_key}"}
|
| 143 |
|
| 144 |
|
| 145 |
-
def _apply_cloudflare_request_quirks(
|
| 146 |
-
body: dict[str, Any], _request: MessagesRequest, thinking_enabled: bool
|
| 147 |
-
) -> None:
|
| 148 |
-
"""Attach Cloudflare Workers AI chat-template thinking control."""
|
| 149 |
-
extra_body = body.setdefault("extra_body", {})
|
| 150 |
-
if not isinstance(extra_body, dict):
|
| 151 |
-
return
|
| 152 |
-
chat_template_kwargs = extra_body.setdefault("chat_template_kwargs", {})
|
| 153 |
-
if isinstance(chat_template_kwargs, dict):
|
| 154 |
-
chat_template_kwargs.setdefault("thinking", thinking_enabled)
|
| 155 |
-
|
| 156 |
-
|
| 157 |
def _cloudflare_reasoning(delta: Any) -> str | None:
|
| 158 |
reasoning = getattr(delta, "reasoning", None)
|
| 159 |
if isinstance(reasoning, str) and reasoning:
|
|
|
|
| 10 |
from free_claude_code.application.errors import ApplicationUnavailableError
|
| 11 |
from free_claude_code.application.model_metadata import ProviderModelInfo
|
| 12 |
from free_claude_code.config.provider_catalog import CLOUDFLARE_AI_REST_ROOT
|
| 13 |
+
from free_claude_code.core.anthropic import ReasoningReplayMode
|
| 14 |
from free_claude_code.providers.base import ProviderConfig
|
| 15 |
from free_claude_code.providers.http import maybe_await_aclose
|
| 16 |
from free_claude_code.providers.model_listing import (
|
|
|
|
| 19 |
model_infos_from_ids,
|
| 20 |
)
|
| 21 |
from free_claude_code.providers.openai_chat import (
|
| 22 |
+
ChatTemplateReasoning,
|
| 23 |
OpenAIChatProfile,
|
| 24 |
OpenAIChatProvider,
|
| 25 |
OpenAIChatRequestPolicy,
|
| 26 |
+
validate_extra_body_does_not_override_canonical_fields,
|
| 27 |
)
|
| 28 |
from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
| 29 |
|
| 30 |
_REQUEST_POLICY = OpenAIChatRequestPolicy(
|
| 31 |
provider_name="CLOUDFLARE",
|
| 32 |
+
reasoning_replay=ReasoningReplayMode.REASONING_CONTENT,
|
| 33 |
include_extra_body=True,
|
| 34 |
+
extra_body_validator=validate_extra_body_does_not_override_canonical_fields,
|
| 35 |
max_tokens_field="max_completion_tokens",
|
| 36 |
)
|
| 37 |
+
_PROFILE = OpenAIChatProfile(_REQUEST_POLICY, ChatTemplateReasoning())
|
| 38 |
|
| 39 |
|
| 40 |
def cloudflare_ai_base_url(api_root: str | None, account_id: str) -> str:
|
|
|
|
| 121 |
finally:
|
| 122 |
await maybe_await_aclose(response)
|
| 123 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 124 |
def _handle_extra_reasoning(
|
| 125 |
+
self, delta: Any, ledger: Any, *, output_reasoning: bool
|
| 126 |
) -> Iterator[str]:
|
| 127 |
"""Map Cloudflare's ``reasoning`` delta field to Anthropic thinking."""
|
| 128 |
reasoning = _cloudflare_reasoning(delta)
|
| 129 |
+
if not output_reasoning or not reasoning:
|
| 130 |
return
|
| 131 |
yield from ledger.ensure_thinking_block()
|
| 132 |
yield ledger.emit_thinking_delta(reasoning)
|
|
|
|
| 135 |
return {"Authorization": f"Bearer {self._api_key}"}
|
| 136 |
|
| 137 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 138 |
def _cloudflare_reasoning(delta: Any) -> str | None:
|
| 139 |
reasoning = getattr(delta, "reasoning", None)
|
| 140 |
if isinstance(reasoning, str) and reasoning:
|
|
@@ -3,18 +3,22 @@
|
|
| 3 |
from typing import Any
|
| 4 |
|
| 5 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
|
|
|
| 6 |
from free_claude_code.providers.base import ProviderConfig
|
| 7 |
from free_claude_code.providers.openai_chat import (
|
|
|
|
| 8 |
OpenAIChatProfile,
|
| 9 |
OpenAIChatProvider,
|
| 10 |
-
OpenAIChatRequestPolicy,
|
| 11 |
usage_int,
|
| 12 |
)
|
| 13 |
from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
| 14 |
|
| 15 |
-
from .compat import build_deepseek_request_body
|
| 16 |
|
| 17 |
-
_PROFILE = OpenAIChatProfile(
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
|
| 20 |
class DeepSeekProvider(OpenAIChatProvider):
|
|
@@ -28,11 +32,14 @@ class DeepSeekProvider(OpenAIChatProvider):
|
|
| 28 |
)
|
| 29 |
|
| 30 |
def _build_request_body(
|
| 31 |
-
self,
|
|
|
|
|
|
|
|
|
|
| 32 |
) -> dict:
|
| 33 |
return build_deepseek_request_body(
|
| 34 |
request,
|
| 35 |
-
|
| 36 |
)
|
| 37 |
|
| 38 |
def _anthropic_usage_fields(self, usage_info: Any) -> dict[str, int]:
|
|
|
|
| 3 |
from typing import Any
|
| 4 |
|
| 5 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
| 6 |
+
from free_claude_code.core.reasoning import DEFAULT_REASONING_POLICY, ReasoningPolicy
|
| 7 |
from free_claude_code.providers.base import ProviderConfig
|
| 8 |
from free_claude_code.providers.openai_chat import (
|
| 9 |
+
NO_REASONING,
|
| 10 |
OpenAIChatProfile,
|
| 11 |
OpenAIChatProvider,
|
|
|
|
| 12 |
usage_int,
|
| 13 |
)
|
| 14 |
from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
| 15 |
|
| 16 |
+
from .compat import DEEPSEEK_REQUEST_POLICY, build_deepseek_request_body
|
| 17 |
|
| 18 |
+
_PROFILE = OpenAIChatProfile(
|
| 19 |
+
DEEPSEEK_REQUEST_POLICY,
|
| 20 |
+
NO_REASONING,
|
| 21 |
+
)
|
| 22 |
|
| 23 |
|
| 24 |
class DeepSeekProvider(OpenAIChatProvider):
|
|
|
|
| 32 |
)
|
| 33 |
|
| 34 |
def _build_request_body(
|
| 35 |
+
self,
|
| 36 |
+
request: MessagesRequest,
|
| 37 |
+
*,
|
| 38 |
+
reasoning: ReasoningPolicy = DEFAULT_REASONING_POLICY,
|
| 39 |
) -> dict:
|
| 40 |
return build_deepseek_request_body(
|
| 41 |
request,
|
| 42 |
+
reasoning=reasoning,
|
| 43 |
)
|
| 44 |
|
| 45 |
def _anthropic_usage_fields(self, usage_info: Any) -> dict[str, int]:
|
|
@@ -8,17 +8,24 @@ from loguru import logger
|
|
| 8 |
from free_claude_code.application.errors import InvalidRequestError
|
| 9 |
from free_claude_code.config.constants import ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS
|
| 10 |
from free_claude_code.core.anthropic import (
|
|
|
|
| 11 |
dump_messages_request,
|
| 12 |
serialize_tool_result_content,
|
| 13 |
)
|
| 14 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
from free_claude_code.providers.openai_chat import (
|
| 16 |
OpenAIChatRequestPolicy,
|
| 17 |
build_openai_chat_request_body,
|
| 18 |
)
|
| 19 |
|
| 20 |
-
|
| 21 |
provider_name="DEEPSEEK",
|
|
|
|
| 22 |
include_extra_body=True,
|
| 23 |
)
|
| 24 |
|
|
@@ -39,7 +46,7 @@ _OMITTED_ATTACHMENT_BLOCK = {"type": "text", "text": _OMITTED_ATTACHMENT_TEXT}
|
|
| 39 |
|
| 40 |
|
| 41 |
def build_deepseek_request_body(
|
| 42 |
-
request_data: MessagesRequest, *,
|
| 43 |
) -> dict:
|
| 44 |
"""Build a DeepSeek Chat Completions body from an Anthropic request."""
|
| 45 |
logger.debug(
|
|
@@ -57,8 +64,8 @@ def build_deepseek_request_body(
|
|
| 57 |
has_tool_history = _has_tool_history(data)
|
| 58 |
has_replayable_tool_thinking = _all_tool_calls_have_replayable_thinking(data)
|
| 59 |
unsafe_tool_followup = has_tool_history and not has_replayable_tool_thinking
|
| 60 |
-
|
| 61 |
-
if
|
| 62 |
if unsafe_tool_followup:
|
| 63 |
logger.debug(
|
| 64 |
"DEEPSEEK_REQUEST: disabling thinking for tool follow-up without "
|
|
@@ -68,6 +75,7 @@ def build_deepseek_request_body(
|
|
| 68 |
len(data.get("tools", [])),
|
| 69 |
)
|
| 70 |
_remove_deepseek_thinking_hints(data)
|
|
|
|
| 71 |
elif has_tool_history:
|
| 72 |
logger.debug(
|
| 73 |
"DEEPSEEK_REQUEST: keeping thinking for tool follow-up with "
|
|
@@ -93,9 +101,8 @@ def build_deepseek_request_body(
|
|
| 93 |
sanitized_request = MessagesRequest.model_validate(data)
|
| 94 |
body = build_openai_chat_request_body(
|
| 95 |
sanitized_request,
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
policy=_REQUEST_POLICY,
|
| 99 |
postprocessors=(_apply_deepseek_chat_extras,),
|
| 100 |
)
|
| 101 |
if "max_tokens" not in body or body.get("max_tokens") is None:
|
|
@@ -427,10 +434,17 @@ def _downgrade_forced_tool_choice(data: dict[str, Any]) -> None:
|
|
| 427 |
|
| 428 |
|
| 429 |
def _apply_deepseek_chat_extras(
|
| 430 |
-
body: dict[str, Any], _request_data: MessagesRequest,
|
| 431 |
) -> None:
|
| 432 |
-
if not thinking_enabled or body.get("model") == "deepseek-reasoner":
|
| 433 |
-
return
|
| 434 |
extra_body = body.setdefault("extra_body", {})
|
| 435 |
-
if isinstance(extra_body, dict):
|
| 436 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
from free_claude_code.application.errors import InvalidRequestError
|
| 9 |
from free_claude_code.config.constants import ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS
|
| 10 |
from free_claude_code.core.anthropic import (
|
| 11 |
+
ReasoningReplayMode,
|
| 12 |
dump_messages_request,
|
| 13 |
serialize_tool_result_content,
|
| 14 |
)
|
| 15 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
| 16 |
+
from free_claude_code.core.reasoning import (
|
| 17 |
+
ReasoningControl,
|
| 18 |
+
ReasoningEffort,
|
| 19 |
+
ReasoningPolicy,
|
| 20 |
+
)
|
| 21 |
from free_claude_code.providers.openai_chat import (
|
| 22 |
OpenAIChatRequestPolicy,
|
| 23 |
build_openai_chat_request_body,
|
| 24 |
)
|
| 25 |
|
| 26 |
+
DEEPSEEK_REQUEST_POLICY = OpenAIChatRequestPolicy(
|
| 27 |
provider_name="DEEPSEEK",
|
| 28 |
+
reasoning_replay=ReasoningReplayMode.REASONING_CONTENT,
|
| 29 |
include_extra_body=True,
|
| 30 |
)
|
| 31 |
|
|
|
|
| 46 |
|
| 47 |
|
| 48 |
def build_deepseek_request_body(
|
| 49 |
+
request_data: MessagesRequest, *, reasoning: ReasoningPolicy
|
| 50 |
) -> dict:
|
| 51 |
"""Build a DeepSeek Chat Completions body from an Anthropic request."""
|
| 52 |
logger.debug(
|
|
|
|
| 64 |
has_tool_history = _has_tool_history(data)
|
| 65 |
has_replayable_tool_thinking = _all_tool_calls_have_replayable_thinking(data)
|
| 66 |
unsafe_tool_followup = has_tool_history and not has_replayable_tool_thinking
|
| 67 |
+
effective_reasoning = reasoning
|
| 68 |
+
if reasoning.control is not ReasoningControl.OFF:
|
| 69 |
if unsafe_tool_followup:
|
| 70 |
logger.debug(
|
| 71 |
"DEEPSEEK_REQUEST: disabling thinking for tool follow-up without "
|
|
|
|
| 75 |
len(data.get("tools", [])),
|
| 76 |
)
|
| 77 |
_remove_deepseek_thinking_hints(data)
|
| 78 |
+
effective_reasoning = ReasoningPolicy.off()
|
| 79 |
elif has_tool_history:
|
| 80 |
logger.debug(
|
| 81 |
"DEEPSEEK_REQUEST: keeping thinking for tool follow-up with "
|
|
|
|
| 101 |
sanitized_request = MessagesRequest.model_validate(data)
|
| 102 |
body = build_openai_chat_request_body(
|
| 103 |
sanitized_request,
|
| 104 |
+
reasoning=effective_reasoning,
|
| 105 |
+
policy=DEEPSEEK_REQUEST_POLICY,
|
|
|
|
| 106 |
postprocessors=(_apply_deepseek_chat_extras,),
|
| 107 |
)
|
| 108 |
if "max_tokens" not in body or body.get("max_tokens") is None:
|
|
|
|
| 434 |
|
| 435 |
|
| 436 |
def _apply_deepseek_chat_extras(
|
| 437 |
+
body: dict[str, Any], _request_data: MessagesRequest, policy: ReasoningPolicy
|
| 438 |
) -> None:
|
|
|
|
|
|
|
| 439 |
extra_body = body.setdefault("extra_body", {})
|
| 440 |
+
if not isinstance(extra_body, dict):
|
| 441 |
+
return
|
| 442 |
+
if policy.control is ReasoningControl.OFF:
|
| 443 |
+
extra_body["thinking"] = {"type": "disabled"}
|
| 444 |
+
return
|
| 445 |
+
if policy.effort in {ReasoningEffort.XHIGH, ReasoningEffort.MAX}:
|
| 446 |
+
body["reasoning_effort"] = "max"
|
| 447 |
+
elif policy.effort is not None:
|
| 448 |
+
body["reasoning_effort"] = "high"
|
| 449 |
+
elif policy.requests_reasoning:
|
| 450 |
+
extra_body["thinking"] = {"type": "enabled"}
|
|
@@ -3,9 +3,16 @@
|
|
| 3 |
from copy import deepcopy
|
| 4 |
from typing import Any
|
| 5 |
|
|
|
|
| 6 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
from free_claude_code.providers.base import ProviderConfig
|
| 8 |
from free_claude_code.providers.openai_chat import (
|
|
|
|
| 9 |
OpenAIChatProfile,
|
| 10 |
OpenAIChatProvider,
|
| 11 |
OpenAIChatRequestPolicy,
|
|
@@ -16,8 +23,24 @@ from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
|
| 16 |
from .quirks import apply_gemini_request_quirks
|
| 17 |
|
| 18 |
_MAX_TOOL_CALL_EXTRA_CONTENT_CACHE = 4096
|
| 19 |
-
_REQUEST_POLICY = OpenAIChatRequestPolicy(
|
| 20 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
|
| 23 |
class GeminiProvider(OpenAIChatProvider):
|
|
@@ -45,18 +68,22 @@ class GeminiProvider(OpenAIChatProvider):
|
|
| 45 |
self._tool_call_extra_content_by_id[tool_call_id] = deepcopy(extra_content)
|
| 46 |
|
| 47 |
def _build_request_body(
|
| 48 |
-
self,
|
|
|
|
|
|
|
|
|
|
| 49 |
) -> dict:
|
| 50 |
return build_openai_chat_request_body(
|
| 51 |
request,
|
| 52 |
-
|
| 53 |
policy=_REQUEST_POLICY,
|
| 54 |
postprocessors=(
|
| 55 |
-
lambda body, request_data,
|
| 56 |
body,
|
| 57 |
request_data,
|
| 58 |
-
|
| 59 |
tool_call_extra_content_by_id=self._tool_call_extra_content_by_id,
|
| 60 |
),
|
|
|
|
| 61 |
),
|
| 62 |
)
|
|
|
|
| 3 |
from copy import deepcopy
|
| 4 |
from typing import Any
|
| 5 |
|
| 6 |
+
from free_claude_code.core.anthropic import ReasoningReplayMode
|
| 7 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
| 8 |
+
from free_claude_code.core.reasoning import (
|
| 9 |
+
DEFAULT_REASONING_POLICY,
|
| 10 |
+
ReasoningEffort,
|
| 11 |
+
ReasoningPolicy,
|
| 12 |
+
)
|
| 13 |
from free_claude_code.providers.base import ProviderConfig
|
| 14 |
from free_claude_code.providers.openai_chat import (
|
| 15 |
+
NamedEffortReasoning,
|
| 16 |
OpenAIChatProfile,
|
| 17 |
OpenAIChatProvider,
|
| 18 |
OpenAIChatRequestPolicy,
|
|
|
|
| 23 |
from .quirks import apply_gemini_request_quirks
|
| 24 |
|
| 25 |
_MAX_TOOL_CALL_EXTRA_CONTENT_CACHE = 4096
|
| 26 |
+
_REQUEST_POLICY = OpenAIChatRequestPolicy(
|
| 27 |
+
provider_name="GEMINI",
|
| 28 |
+
reasoning_replay=ReasoningReplayMode.REASONING_CONTENT,
|
| 29 |
+
)
|
| 30 |
+
_PROFILE = OpenAIChatProfile(
|
| 31 |
+
_REQUEST_POLICY,
|
| 32 |
+
NamedEffortReasoning(
|
| 33 |
+
(
|
| 34 |
+
(ReasoningEffort.MINIMAL, "minimal"),
|
| 35 |
+
(ReasoningEffort.LOW, "low"),
|
| 36 |
+
(ReasoningEffort.MEDIUM, "medium"),
|
| 37 |
+
(ReasoningEffort.HIGH, "high"),
|
| 38 |
+
(ReasoningEffort.XHIGH, "high"),
|
| 39 |
+
(ReasoningEffort.MAX, "high"),
|
| 40 |
+
),
|
| 41 |
+
disabled_value="none",
|
| 42 |
+
),
|
| 43 |
+
)
|
| 44 |
|
| 45 |
|
| 46 |
class GeminiProvider(OpenAIChatProvider):
|
|
|
|
| 68 |
self._tool_call_extra_content_by_id[tool_call_id] = deepcopy(extra_content)
|
| 69 |
|
| 70 |
def _build_request_body(
|
| 71 |
+
self,
|
| 72 |
+
request: MessagesRequest,
|
| 73 |
+
*,
|
| 74 |
+
reasoning: ReasoningPolicy = DEFAULT_REASONING_POLICY,
|
| 75 |
) -> dict:
|
| 76 |
return build_openai_chat_request_body(
|
| 77 |
request,
|
| 78 |
+
reasoning=reasoning,
|
| 79 |
policy=_REQUEST_POLICY,
|
| 80 |
postprocessors=(
|
| 81 |
+
lambda body, request_data, policy: apply_gemini_request_quirks(
|
| 82 |
body,
|
| 83 |
request_data,
|
| 84 |
+
policy,
|
| 85 |
tool_call_extra_content_by_id=self._tool_call_extra_content_by_id,
|
| 86 |
),
|
| 87 |
+
_PROFILE.apply_reasoning,
|
| 88 |
),
|
| 89 |
)
|
|
@@ -4,6 +4,7 @@ from copy import deepcopy
|
|
| 4 |
from typing import Any, cast
|
| 5 |
|
| 6 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
|
|
|
| 7 |
|
| 8 |
GEMINI_SKIP_THOUGHT_SIGNATURE_VALIDATOR = "skip_thought_signature_validator"
|
| 9 |
|
|
@@ -11,7 +12,7 @@ GEMINI_SKIP_THOUGHT_SIGNATURE_VALIDATOR = "skip_thought_signature_validator"
|
|
| 11 |
def apply_gemini_request_quirks(
|
| 12 |
body: dict[str, Any],
|
| 13 |
request_data: MessagesRequest,
|
| 14 |
-
|
| 15 |
*,
|
| 16 |
tool_call_extra_content_by_id: dict[str, dict[str, Any]] | None = None,
|
| 17 |
) -> None:
|
|
@@ -21,10 +22,8 @@ def apply_gemini_request_quirks(
|
|
| 21 |
if isinstance(request_extra, dict):
|
| 22 |
extra_body.update(deepcopy(request_extra))
|
| 23 |
|
| 24 |
-
if
|
| 25 |
_apply_thinking_config(extra_body)
|
| 26 |
-
else:
|
| 27 |
-
body["reasoning_effort"] = "none"
|
| 28 |
|
| 29 |
if extra_body:
|
| 30 |
body["extra_body"] = extra_body
|
|
@@ -53,10 +52,6 @@ def _apply_thinking_config(extra_body: dict[str, Any]) -> None:
|
|
| 53 |
thinking_cfg.setdefault("include_thoughts", True)
|
| 54 |
|
| 55 |
|
| 56 |
-
def _is_gemini_3_model(model: Any) -> bool:
|
| 57 |
-
return "gemini-3" in str(model).lower()
|
| 58 |
-
|
| 59 |
-
|
| 60 |
def _thought_signature_from_extra_content(extra_content: Any) -> str | None:
|
| 61 |
if not isinstance(extra_content, dict):
|
| 62 |
return None
|
|
@@ -136,12 +131,7 @@ def _apply_cached_tool_call_signatures(
|
|
| 136 |
tool_call["extra_content"] = deepcopy(cached_extra_content)
|
| 137 |
|
| 138 |
|
| 139 |
-
def
|
| 140 |
-
body: dict[str, Any], messages: list[Any]
|
| 141 |
-
) -> None:
|
| 142 |
-
if not _is_gemini_3_model(body.get("model")):
|
| 143 |
-
return
|
| 144 |
-
|
| 145 |
start_index = _current_turn_start_index(messages)
|
| 146 |
for message in messages[start_index + 1 :]:
|
| 147 |
if not isinstance(message, dict) or message.get("role") != "assistant":
|
|
@@ -168,4 +158,4 @@ def _apply_gemini_tool_call_signatures(
|
|
| 168 |
if not isinstance(messages, list):
|
| 169 |
return
|
| 170 |
_apply_cached_tool_call_signatures(messages, tool_call_extra_content_by_id or {})
|
| 171 |
-
|
|
|
|
| 4 |
from typing import Any, cast
|
| 5 |
|
| 6 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
| 7 |
+
from free_claude_code.core.reasoning import ReasoningPolicy
|
| 8 |
|
| 9 |
GEMINI_SKIP_THOUGHT_SIGNATURE_VALIDATOR = "skip_thought_signature_validator"
|
| 10 |
|
|
|
|
| 12 |
def apply_gemini_request_quirks(
|
| 13 |
body: dict[str, Any],
|
| 14 |
request_data: MessagesRequest,
|
| 15 |
+
reasoning: ReasoningPolicy,
|
| 16 |
*,
|
| 17 |
tool_call_extra_content_by_id: dict[str, dict[str, Any]] | None = None,
|
| 18 |
) -> None:
|
|
|
|
| 22 |
if isinstance(request_extra, dict):
|
| 23 |
extra_body.update(deepcopy(request_extra))
|
| 24 |
|
| 25 |
+
if reasoning.requests_reasoning:
|
| 26 |
_apply_thinking_config(extra_body)
|
|
|
|
|
|
|
| 27 |
|
| 28 |
if extra_body:
|
| 29 |
body["extra_body"] = extra_body
|
|
|
|
| 52 |
thinking_cfg.setdefault("include_thoughts", True)
|
| 53 |
|
| 54 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
def _thought_signature_from_extra_content(extra_content: Any) -> str | None:
|
| 56 |
if not isinstance(extra_content, dict):
|
| 57 |
return None
|
|
|
|
| 131 |
tool_call["extra_content"] = deepcopy(cached_extra_content)
|
| 132 |
|
| 133 |
|
| 134 |
+
def _apply_missing_current_turn_signatures(messages: list[Any]) -> None:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 135 |
start_index = _current_turn_start_index(messages)
|
| 136 |
for message in messages[start_index + 1 :]:
|
| 137 |
if not isinstance(message, dict) or message.get("role") != "assistant":
|
|
|
|
| 158 |
if not isinstance(messages, list):
|
| 159 |
return
|
| 160 |
_apply_cached_tool_call_signatures(messages, tool_call_extra_content_by_id or {})
|
| 161 |
+
_apply_missing_current_turn_signatures(messages)
|
|
@@ -6,7 +6,7 @@ from typing import Any
|
|
| 6 |
import httpx
|
| 7 |
|
| 8 |
from free_claude_code.application.model_metadata import ProviderModelInfo
|
| 9 |
-
from free_claude_code.core.anthropic
|
| 10 |
from free_claude_code.providers.base import ProviderConfig
|
| 11 |
from free_claude_code.providers.http import maybe_await_aclose
|
| 12 |
from free_claude_code.providers.model_listing import (
|
|
@@ -14,10 +14,10 @@ from free_claude_code.providers.model_listing import (
|
|
| 14 |
model_infos_from_ids,
|
| 15 |
)
|
| 16 |
from free_claude_code.providers.openai_chat import (
|
|
|
|
| 17 |
OpenAIChatProfile,
|
| 18 |
OpenAIChatProvider,
|
| 19 |
OpenAIChatRequestPolicy,
|
| 20 |
-
build_openai_chat_request_body,
|
| 21 |
)
|
| 22 |
from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
| 23 |
|
|
@@ -26,8 +26,9 @@ GITHUB_MODELS_API_VERSION = "2026-03-10"
|
|
| 26 |
|
| 27 |
_REQUEST_POLICY = OpenAIChatRequestPolicy(
|
| 28 |
provider_name="GITHUB_MODELS",
|
|
|
|
| 29 |
)
|
| 30 |
-
_PROFILE = OpenAIChatProfile(_REQUEST_POLICY)
|
| 31 |
_REQUIRED_MODEL_CAPABILITIES = frozenset({"streaming", "tool-calling"})
|
| 32 |
|
| 33 |
|
|
@@ -81,15 +82,6 @@ class GitHubModelsProvider(OpenAIChatProvider):
|
|
| 81 |
finally:
|
| 82 |
await maybe_await_aclose(response)
|
| 83 |
|
| 84 |
-
def _build_request_body(
|
| 85 |
-
self, request: MessagesRequest, thinking_enabled: bool | None = None
|
| 86 |
-
) -> dict:
|
| 87 |
-
return build_openai_chat_request_body(
|
| 88 |
-
request,
|
| 89 |
-
thinking_enabled=self._is_thinking_enabled(request, thinking_enabled),
|
| 90 |
-
policy=_REQUEST_POLICY,
|
| 91 |
-
)
|
| 92 |
-
|
| 93 |
def _model_list_headers(self) -> dict[str, str]:
|
| 94 |
return _github_models_api_headers(self._api_key)
|
| 95 |
|
|
|
|
| 6 |
import httpx
|
| 7 |
|
| 8 |
from free_claude_code.application.model_metadata import ProviderModelInfo
|
| 9 |
+
from free_claude_code.core.anthropic import ReasoningReplayMode
|
| 10 |
from free_claude_code.providers.base import ProviderConfig
|
| 11 |
from free_claude_code.providers.http import maybe_await_aclose
|
| 12 |
from free_claude_code.providers.model_listing import (
|
|
|
|
| 14 |
model_infos_from_ids,
|
| 15 |
)
|
| 16 |
from free_claude_code.providers.openai_chat import (
|
| 17 |
+
NO_REASONING,
|
| 18 |
OpenAIChatProfile,
|
| 19 |
OpenAIChatProvider,
|
| 20 |
OpenAIChatRequestPolicy,
|
|
|
|
| 21 |
)
|
| 22 |
from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
| 23 |
|
|
|
|
| 26 |
|
| 27 |
_REQUEST_POLICY = OpenAIChatRequestPolicy(
|
| 28 |
provider_name="GITHUB_MODELS",
|
| 29 |
+
reasoning_replay=ReasoningReplayMode.THINK_TAGS,
|
| 30 |
)
|
| 31 |
+
_PROFILE = OpenAIChatProfile(_REQUEST_POLICY, NO_REASONING)
|
| 32 |
_REQUIRED_MODEL_CAPABILITIES = frozenset({"streaming", "tool-calling"})
|
| 33 |
|
| 34 |
|
|
|
|
| 82 |
finally:
|
| 83 |
await maybe_await_aclose(response)
|
| 84 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
def _model_list_headers(self) -> dict[str, str]:
|
| 86 |
return _github_models_api_headers(self._api_key)
|
| 87 |
|
|
@@ -17,22 +17,41 @@ import httpx
|
|
| 17 |
from loguru import logger
|
| 18 |
|
| 19 |
from free_claude_code.application.errors import InvalidRequestError
|
| 20 |
-
from free_claude_code.core.anthropic import
|
| 21 |
-
ReasoningReplayMode,
|
| 22 |
-
build_base_request_body,
|
| 23 |
-
get_token_count,
|
| 24 |
-
)
|
| 25 |
-
from free_claude_code.core.anthropic.conversion import OpenAIConversionError
|
| 26 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
from free_claude_code.providers.base import ProviderConfig
|
| 28 |
from free_claude_code.providers.openai_chat import (
|
|
|
|
| 29 |
OpenAIChatProfile,
|
| 30 |
OpenAIChatProvider,
|
| 31 |
OpenAIChatRequestPolicy,
|
| 32 |
)
|
| 33 |
from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
| 34 |
|
| 35 |
-
_PROFILE = OpenAIChatProfile(
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
|
| 38 |
class LMStudioProvider(OpenAIChatProvider):
|
|
@@ -53,28 +72,13 @@ class LMStudioProvider(OpenAIChatProvider):
|
|
| 53 |
)
|
| 54 |
self._loaded_context_cache: tuple[float, int | None] = (0.0, None)
|
| 55 |
|
| 56 |
-
def _build_request_body(
|
| 57 |
-
self, request: MessagesRequest, thinking_enabled: bool | None = None
|
| 58 |
-
) -> dict:
|
| 59 |
-
"""Build an OpenAI chat body from the Anthropic request.
|
| 60 |
-
|
| 61 |
-
Prior-turn thinking is never replayed: Mistral-family templates have
|
| 62 |
-
no assistant reasoning field, and replaying ``<think>`` text inflates
|
| 63 |
-
the local context for no benefit. New-response thinking still streams
|
| 64 |
-
back via ``reasoning_content``/``<think>`` parsing in the provider.
|
| 65 |
-
"""
|
| 66 |
-
try:
|
| 67 |
-
return build_base_request_body(
|
| 68 |
-
request,
|
| 69 |
-
reasoning_replay=ReasoningReplayMode.DISABLED,
|
| 70 |
-
)
|
| 71 |
-
except OpenAIConversionError as exc:
|
| 72 |
-
raise InvalidRequestError(str(exc)) from exc
|
| 73 |
-
|
| 74 |
def preflight_stream(
|
| 75 |
-
self,
|
|
|
|
|
|
|
|
|
|
| 76 |
) -> None:
|
| 77 |
-
super().preflight_stream(request,
|
| 78 |
self._preflight_context_budget(request)
|
| 79 |
|
| 80 |
def _preflight_context_budget(self, request: MessagesRequest) -> None:
|
|
|
|
| 17 |
from loguru import logger
|
| 18 |
|
| 19 |
from free_claude_code.application.errors import InvalidRequestError
|
| 20 |
+
from free_claude_code.core.anthropic import ReasoningReplayMode, get_token_count
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
| 22 |
+
from free_claude_code.core.reasoning import (
|
| 23 |
+
DEFAULT_REASONING_POLICY,
|
| 24 |
+
ReasoningEffort,
|
| 25 |
+
ReasoningPolicy,
|
| 26 |
+
)
|
| 27 |
from free_claude_code.providers.base import ProviderConfig
|
| 28 |
from free_claude_code.providers.openai_chat import (
|
| 29 |
+
NamedEffortReasoning,
|
| 30 |
OpenAIChatProfile,
|
| 31 |
OpenAIChatProvider,
|
| 32 |
OpenAIChatRequestPolicy,
|
| 33 |
)
|
| 34 |
from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
| 35 |
|
| 36 |
+
_PROFILE = OpenAIChatProfile(
|
| 37 |
+
OpenAIChatRequestPolicy(
|
| 38 |
+
provider_name="LMSTUDIO",
|
| 39 |
+
reasoning_replay=ReasoningReplayMode.DISABLED,
|
| 40 |
+
),
|
| 41 |
+
NamedEffortReasoning(
|
| 42 |
+
(
|
| 43 |
+
(ReasoningEffort.MINIMAL, "low"),
|
| 44 |
+
(ReasoningEffort.LOW, "low"),
|
| 45 |
+
(ReasoningEffort.MEDIUM, "medium"),
|
| 46 |
+
(ReasoningEffort.HIGH, "high"),
|
| 47 |
+
(ReasoningEffort.XHIGH, "high"),
|
| 48 |
+
(ReasoningEffort.MAX, "high"),
|
| 49 |
+
),
|
| 50 |
+
disabled_value="none",
|
| 51 |
+
enabled_value="high",
|
| 52 |
+
budget_field="reasoning_tokens",
|
| 53 |
+
),
|
| 54 |
+
)
|
| 55 |
|
| 56 |
|
| 57 |
class LMStudioProvider(OpenAIChatProvider):
|
|
|
|
| 72 |
)
|
| 73 |
self._loaded_context_cache: tuple[float, int | None] = (0.0, None)
|
| 74 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
def preflight_stream(
|
| 76 |
+
self,
|
| 77 |
+
request: MessagesRequest,
|
| 78 |
+
*,
|
| 79 |
+
reasoning: ReasoningPolicy = DEFAULT_REASONING_POLICY,
|
| 80 |
) -> None:
|
| 81 |
+
super().preflight_stream(request, reasoning=reasoning)
|
| 82 |
self._preflight_context_budget(request)
|
| 83 |
|
| 84 |
def _preflight_context_budget(self, request: MessagesRequest) -> None:
|
|
@@ -4,9 +4,12 @@ from typing import Any
|
|
| 4 |
|
| 5 |
from loguru import logger
|
| 6 |
|
|
|
|
| 7 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
|
|
|
| 8 |
from free_claude_code.providers.base import ProviderConfig
|
| 9 |
from free_claude_code.providers.openai_chat import (
|
|
|
|
| 10 |
OpenAIChatProfile,
|
| 11 |
OpenAIChatProvider,
|
| 12 |
OpenAIChatRequestPolicy,
|
|
@@ -21,8 +24,11 @@ from .reasoning import (
|
|
| 21 |
normalize_mistral_stream,
|
| 22 |
)
|
| 23 |
|
| 24 |
-
_REQUEST_POLICY = OpenAIChatRequestPolicy(
|
| 25 |
-
|
|
|
|
|
|
|
|
|
|
| 26 |
|
| 27 |
|
| 28 |
class MistralProvider(OpenAIChatProvider):
|
|
@@ -36,19 +42,17 @@ class MistralProvider(OpenAIChatProvider):
|
|
| 36 |
)
|
| 37 |
|
| 38 |
def _build_request_body(
|
| 39 |
-
self,
|
|
|
|
|
|
|
|
|
|
| 40 |
) -> dict:
|
| 41 |
-
effective_thinking_enabled = self._is_thinking_enabled(
|
| 42 |
-
request, thinking_enabled
|
| 43 |
-
)
|
| 44 |
body = build_openai_chat_request_body(
|
| 45 |
request,
|
| 46 |
-
|
| 47 |
policy=_REQUEST_POLICY,
|
| 48 |
)
|
| 49 |
-
apply_mistral_reasoning_request_shape(
|
| 50 |
-
body, thinking_enabled=effective_thinking_enabled
|
| 51 |
-
)
|
| 52 |
return body
|
| 53 |
|
| 54 |
def _get_retry_request_body(self, error: Exception, body: dict) -> dict | None:
|
|
|
|
| 4 |
|
| 5 |
from loguru import logger
|
| 6 |
|
| 7 |
+
from free_claude_code.core.anthropic import ReasoningReplayMode
|
| 8 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
| 9 |
+
from free_claude_code.core.reasoning import DEFAULT_REASONING_POLICY, ReasoningPolicy
|
| 10 |
from free_claude_code.providers.base import ProviderConfig
|
| 11 |
from free_claude_code.providers.openai_chat import (
|
| 12 |
+
NO_REASONING,
|
| 13 |
OpenAIChatProfile,
|
| 14 |
OpenAIChatProvider,
|
| 15 |
OpenAIChatRequestPolicy,
|
|
|
|
| 24 |
normalize_mistral_stream,
|
| 25 |
)
|
| 26 |
|
| 27 |
+
_REQUEST_POLICY = OpenAIChatRequestPolicy(
|
| 28 |
+
provider_name="MISTRAL",
|
| 29 |
+
reasoning_replay=ReasoningReplayMode.REASONING_CONTENT,
|
| 30 |
+
)
|
| 31 |
+
_PROFILE = OpenAIChatProfile(_REQUEST_POLICY, NO_REASONING)
|
| 32 |
|
| 33 |
|
| 34 |
class MistralProvider(OpenAIChatProvider):
|
|
|
|
| 42 |
)
|
| 43 |
|
| 44 |
def _build_request_body(
|
| 45 |
+
self,
|
| 46 |
+
request: MessagesRequest,
|
| 47 |
+
*,
|
| 48 |
+
reasoning: ReasoningPolicy = DEFAULT_REASONING_POLICY,
|
| 49 |
) -> dict:
|
|
|
|
|
|
|
|
|
|
| 50 |
body = build_openai_chat_request_body(
|
| 51 |
request,
|
| 52 |
+
reasoning=reasoning,
|
| 53 |
policy=_REQUEST_POLICY,
|
| 54 |
)
|
| 55 |
+
apply_mistral_reasoning_request_shape(body, reasoning=reasoning)
|
|
|
|
|
|
|
| 56 |
return body
|
| 57 |
|
| 58 |
def _get_retry_request_body(self, error: Exception, body: dict) -> dict | None:
|
|
@@ -8,6 +8,7 @@ from typing import Any
|
|
| 8 |
|
| 9 |
import openai
|
| 10 |
|
|
|
|
| 11 |
from free_claude_code.providers.http import maybe_await_aclose
|
| 12 |
|
| 13 |
MISTRAL_REASONING_EFFORT = "high"
|
|
@@ -23,10 +24,12 @@ _REJECTION_WORDS = ("unsupported", "unknown", "invalid", "forbidden", "extra")
|
|
| 23 |
|
| 24 |
|
| 25 |
def apply_mistral_reasoning_request_shape(
|
| 26 |
-
body: dict[str, Any], *,
|
| 27 |
) -> None:
|
| 28 |
"""Apply Mistral's native reasoning request shape in-place."""
|
| 29 |
-
if
|
|
|
|
|
|
|
| 30 |
body["reasoning_effort"] = MISTRAL_REASONING_EFFORT
|
| 31 |
else:
|
| 32 |
body.pop("reasoning_effort", None)
|
|
@@ -38,13 +41,11 @@ def apply_mistral_reasoning_request_shape(
|
|
| 38 |
for message in messages:
|
| 39 |
if not isinstance(message, dict) or message.get("role") != "assistant":
|
| 40 |
continue
|
| 41 |
-
|
| 42 |
-
if
|
| 43 |
message["content"] = _content_with_prepended_thinking(
|
| 44 |
-
message.get("content"),
|
| 45 |
)
|
| 46 |
-
elif not thinking_enabled:
|
| 47 |
-
message["content"] = _content_without_thinking(message.get("content"))
|
| 48 |
|
| 49 |
|
| 50 |
def clone_body_without_mistral_reasoning(
|
|
@@ -178,11 +179,6 @@ def _content_with_prepended_thinking(
|
|
| 178 |
return chunks
|
| 179 |
|
| 180 |
|
| 181 |
-
def _content_without_thinking(content: Any) -> Any:
|
| 182 |
-
stripped, _ = _strip_mistral_thinking_content(content)
|
| 183 |
-
return stripped
|
| 184 |
-
|
| 185 |
-
|
| 186 |
def _strip_mistral_thinking_content(content: Any) -> tuple[Any, bool]:
|
| 187 |
if not _is_sequence(content):
|
| 188 |
return content, False
|
|
|
|
| 8 |
|
| 9 |
import openai
|
| 10 |
|
| 11 |
+
from free_claude_code.core.reasoning import ReasoningControl, ReasoningPolicy
|
| 12 |
from free_claude_code.providers.http import maybe_await_aclose
|
| 13 |
|
| 14 |
MISTRAL_REASONING_EFFORT = "high"
|
|
|
|
| 24 |
|
| 25 |
|
| 26 |
def apply_mistral_reasoning_request_shape(
|
| 27 |
+
body: dict[str, Any], *, reasoning: ReasoningPolicy
|
| 28 |
) -> None:
|
| 29 |
"""Apply Mistral's native reasoning request shape in-place."""
|
| 30 |
+
if reasoning.control is ReasoningControl.OFF:
|
| 31 |
+
body["reasoning_effort"] = "none"
|
| 32 |
+
elif reasoning.requests_reasoning:
|
| 33 |
body["reasoning_effort"] = MISTRAL_REASONING_EFFORT
|
| 34 |
else:
|
| 35 |
body.pop("reasoning_effort", None)
|
|
|
|
| 41 |
for message in messages:
|
| 42 |
if not isinstance(message, dict) or message.get("role") != "assistant":
|
| 43 |
continue
|
| 44 |
+
replayed_reasoning = _clean_text(message.pop("reasoning_content", None))
|
| 45 |
+
if replayed_reasoning:
|
| 46 |
message["content"] = _content_with_prepended_thinking(
|
| 47 |
+
message.get("content"), replayed_reasoning
|
| 48 |
)
|
|
|
|
|
|
|
| 49 |
|
| 50 |
|
| 51 |
def clone_body_without_mistral_reasoning(
|
|
|
|
| 179 |
return chunks
|
| 180 |
|
| 181 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 182 |
def _strip_mistral_thinking_content(content: Any) -> tuple[Any, bool]:
|
| 183 |
if not _is_sequence(content):
|
| 184 |
return content, False
|
|
@@ -10,18 +10,19 @@ from loguru import logger
|
|
| 10 |
from free_claude_code.config.nim import NimSettings
|
| 11 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
| 12 |
from free_claude_code.core.failures import ExecutionFailure
|
|
|
|
| 13 |
from free_claude_code.providers.base import ProviderConfig
|
| 14 |
from free_claude_code.providers.failure_policy import (
|
| 15 |
overloaded_provider_failure,
|
| 16 |
)
|
| 17 |
from free_claude_code.providers.openai_chat import (
|
|
|
|
| 18 |
OpenAIChatProfile,
|
| 19 |
OpenAIChatProvider,
|
| 20 |
-
OpenAIChatRequestPolicy,
|
| 21 |
)
|
| 22 |
from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
| 23 |
|
| 24 |
-
from .request_options import build_nim_request_body
|
| 25 |
from .retry import (
|
| 26 |
clone_body_without_chat_template,
|
| 27 |
clone_body_without_reasoning_budget,
|
|
@@ -33,7 +34,10 @@ from .tool_schema import (
|
|
| 33 |
)
|
| 34 |
|
| 35 |
_DEGRADED_FUNCTION_STATE = "degraded function cannot be invoked"
|
| 36 |
-
_PROFILE = OpenAIChatProfile(
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
|
| 39 |
class NvidiaNimProvider(OpenAIChatProvider):
|
|
@@ -54,13 +58,16 @@ class NvidiaNimProvider(OpenAIChatProvider):
|
|
| 54 |
self._nim_settings = nim_settings
|
| 55 |
|
| 56 |
def _build_request_body(
|
| 57 |
-
self,
|
|
|
|
|
|
|
|
|
|
| 58 |
) -> dict:
|
| 59 |
"""Internal helper for tests and shared building."""
|
| 60 |
return build_nim_request_body(
|
| 61 |
request,
|
| 62 |
self._nim_settings,
|
| 63 |
-
|
| 64 |
)
|
| 65 |
|
| 66 |
def _prepare_create_body(self, body: dict[str, Any]) -> dict[str, Any]:
|
|
|
|
| 10 |
from free_claude_code.config.nim import NimSettings
|
| 11 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
| 12 |
from free_claude_code.core.failures import ExecutionFailure
|
| 13 |
+
from free_claude_code.core.reasoning import DEFAULT_REASONING_POLICY, ReasoningPolicy
|
| 14 |
from free_claude_code.providers.base import ProviderConfig
|
| 15 |
from free_claude_code.providers.failure_policy import (
|
| 16 |
overloaded_provider_failure,
|
| 17 |
)
|
| 18 |
from free_claude_code.providers.openai_chat import (
|
| 19 |
+
NO_REASONING,
|
| 20 |
OpenAIChatProfile,
|
| 21 |
OpenAIChatProvider,
|
|
|
|
| 22 |
)
|
| 23 |
from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
| 24 |
|
| 25 |
+
from .request_options import NIM_REQUEST_POLICY, build_nim_request_body
|
| 26 |
from .retry import (
|
| 27 |
clone_body_without_chat_template,
|
| 28 |
clone_body_without_reasoning_budget,
|
|
|
|
| 34 |
)
|
| 35 |
|
| 36 |
_DEGRADED_FUNCTION_STATE = "degraded function cannot be invoked"
|
| 37 |
+
_PROFILE = OpenAIChatProfile(
|
| 38 |
+
NIM_REQUEST_POLICY,
|
| 39 |
+
NO_REASONING,
|
| 40 |
+
)
|
| 41 |
|
| 42 |
|
| 43 |
class NvidiaNimProvider(OpenAIChatProvider):
|
|
|
|
| 58 |
self._nim_settings = nim_settings
|
| 59 |
|
| 60 |
def _build_request_body(
|
| 61 |
+
self,
|
| 62 |
+
request: MessagesRequest,
|
| 63 |
+
*,
|
| 64 |
+
reasoning: ReasoningPolicy = DEFAULT_REASONING_POLICY,
|
| 65 |
) -> dict:
|
| 66 |
"""Internal helper for tests and shared building."""
|
| 67 |
return build_nim_request_body(
|
| 68 |
request,
|
| 69 |
self._nim_settings,
|
| 70 |
+
reasoning=reasoning,
|
| 71 |
)
|
| 72 |
|
| 73 |
def _prepare_create_body(self, body: dict[str, Any]) -> dict[str, Any]:
|
|
@@ -1,10 +1,12 @@
|
|
| 1 |
"""NVIDIA NIM request option injection."""
|
| 2 |
|
|
|
|
| 3 |
from typing import Any
|
| 4 |
|
| 5 |
from free_claude_code.config.nim import NimSettings
|
| 6 |
-
from free_claude_code.core.anthropic import set_if_not_none
|
| 7 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
|
|
|
| 8 |
from free_claude_code.providers.openai_chat import (
|
| 9 |
OpenAIChatRequestPolicy,
|
| 10 |
build_openai_chat_request_body,
|
|
@@ -12,22 +14,25 @@ from free_claude_code.providers.openai_chat import (
|
|
| 12 |
|
| 13 |
from .tool_schema import sanitize_nim_tool_schemas
|
| 14 |
|
| 15 |
-
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
|
| 18 |
def build_nim_request_body(
|
| 19 |
-
request_data: MessagesRequest, nim: NimSettings, *,
|
| 20 |
) -> dict[str, Any]:
|
| 21 |
"""Build OpenAI-format request body from Anthropic request plus NIM settings."""
|
| 22 |
return build_openai_chat_request_body(
|
| 23 |
request_data,
|
| 24 |
-
|
| 25 |
-
policy=
|
| 26 |
postprocessors=(
|
| 27 |
-
lambda body, request,
|
| 28 |
body,
|
| 29 |
request,
|
| 30 |
-
|
| 31 |
nim=nim,
|
| 32 |
),
|
| 33 |
),
|
|
@@ -37,7 +42,7 @@ def build_nim_request_body(
|
|
| 37 |
def apply_nim_request_options(
|
| 38 |
body: dict[str, Any],
|
| 39 |
request_data: MessagesRequest,
|
| 40 |
-
|
| 41 |
*,
|
| 42 |
nim: NimSettings,
|
| 43 |
) -> None:
|
|
@@ -71,14 +76,30 @@ def apply_nim_request_options(
|
|
| 71 |
extra_body: dict[str, Any] = {}
|
| 72 |
request_extra = request_data.extra_body
|
| 73 |
if request_extra:
|
| 74 |
-
extra_body.update(request_extra)
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
if isinstance(chat_template_kwargs, dict):
|
| 81 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 82 |
|
| 83 |
req_top_k = request_data.top_k
|
| 84 |
top_k = req_top_k if req_top_k is not None else nim.top_k
|
|
|
|
| 1 |
"""NVIDIA NIM request option injection."""
|
| 2 |
|
| 3 |
+
from copy import deepcopy
|
| 4 |
from typing import Any
|
| 5 |
|
| 6 |
from free_claude_code.config.nim import NimSettings
|
| 7 |
+
from free_claude_code.core.anthropic import ReasoningReplayMode, set_if_not_none
|
| 8 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
| 9 |
+
from free_claude_code.core.reasoning import ReasoningControl, ReasoningPolicy
|
| 10 |
from free_claude_code.providers.openai_chat import (
|
| 11 |
OpenAIChatRequestPolicy,
|
| 12 |
build_openai_chat_request_body,
|
|
|
|
| 14 |
|
| 15 |
from .tool_schema import sanitize_nim_tool_schemas
|
| 16 |
|
| 17 |
+
NIM_REQUEST_POLICY = OpenAIChatRequestPolicy(
|
| 18 |
+
provider_name="NIM",
|
| 19 |
+
reasoning_replay=ReasoningReplayMode.REASONING_CONTENT,
|
| 20 |
+
)
|
| 21 |
|
| 22 |
|
| 23 |
def build_nim_request_body(
|
| 24 |
+
request_data: MessagesRequest, nim: NimSettings, *, reasoning: ReasoningPolicy
|
| 25 |
) -> dict[str, Any]:
|
| 26 |
"""Build OpenAI-format request body from Anthropic request plus NIM settings."""
|
| 27 |
return build_openai_chat_request_body(
|
| 28 |
request_data,
|
| 29 |
+
reasoning=reasoning,
|
| 30 |
+
policy=NIM_REQUEST_POLICY,
|
| 31 |
postprocessors=(
|
| 32 |
+
lambda body, request, policy: apply_nim_request_options(
|
| 33 |
body,
|
| 34 |
request,
|
| 35 |
+
policy,
|
| 36 |
nim=nim,
|
| 37 |
),
|
| 38 |
),
|
|
|
|
| 42 |
def apply_nim_request_options(
|
| 43 |
body: dict[str, Any],
|
| 44 |
request_data: MessagesRequest,
|
| 45 |
+
reasoning: ReasoningPolicy,
|
| 46 |
*,
|
| 47 |
nim: NimSettings,
|
| 48 |
) -> None:
|
|
|
|
| 76 |
extra_body: dict[str, Any] = {}
|
| 77 |
request_extra = request_data.extra_body
|
| 78 |
if request_extra:
|
| 79 |
+
extra_body.update(deepcopy(request_extra))
|
| 80 |
+
for key in (
|
| 81 |
+
"reasoning",
|
| 82 |
+
"reasoning_effort",
|
| 83 |
+
"reasoning_tokens",
|
| 84 |
+
"thinking",
|
| 85 |
+
"thinking_budget_tokens",
|
| 86 |
+
):
|
| 87 |
+
extra_body.pop(key, None)
|
| 88 |
+
request_template_kwargs = extra_body.get("chat_template_kwargs")
|
| 89 |
+
if isinstance(request_template_kwargs, dict):
|
| 90 |
+
for key in ("thinking", "enable_thinking", "reasoning_budget"):
|
| 91 |
+
request_template_kwargs.pop(key, None)
|
| 92 |
+
if not request_template_kwargs:
|
| 93 |
+
extra_body.pop("chat_template_kwargs", None)
|
| 94 |
+
|
| 95 |
+
if reasoning.control is ReasoningControl.OFF or reasoning.requests_reasoning:
|
| 96 |
+
chat_template_kwargs = extra_body.setdefault("chat_template_kwargs", {})
|
| 97 |
if isinstance(chat_template_kwargs, dict):
|
| 98 |
+
enabled = reasoning.control is not ReasoningControl.OFF
|
| 99 |
+
chat_template_kwargs["thinking"] = enabled
|
| 100 |
+
chat_template_kwargs["enable_thinking"] = enabled
|
| 101 |
+
if enabled and reasoning.budget_tokens is not None:
|
| 102 |
+
chat_template_kwargs["reasoning_budget"] = reasoning.budget_tokens
|
| 103 |
|
| 104 |
req_top_k = request_data.top_k
|
| 105 |
top_k = req_top_k if req_top_k is not None else nim.top_k
|
|
@@ -6,8 +6,10 @@ from typing import Any
|
|
| 6 |
|
| 7 |
from free_claude_code.application.model_metadata import ProviderModelInfo
|
| 8 |
from free_claude_code.config.constants import ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS
|
| 9 |
-
from free_claude_code.core.anthropic
|
|
|
|
| 10 |
from free_claude_code.core.anthropic.streaming import AnthropicStreamLedger
|
|
|
|
| 11 |
from free_claude_code.providers.base import ProviderConfig
|
| 12 |
from free_claude_code.providers.model_listing import (
|
| 13 |
extract_openrouter_tool_model_ids,
|
|
@@ -17,18 +19,18 @@ from free_claude_code.providers.openai_chat import (
|
|
| 17 |
OpenAIChatProfile,
|
| 18 |
OpenAIChatProvider,
|
| 19 |
OpenAIChatRequestPolicy,
|
| 20 |
-
|
| 21 |
validate_extra_body_does_not_override_canonical_fields,
|
| 22 |
)
|
| 23 |
from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
| 24 |
|
| 25 |
_REQUEST_POLICY = OpenAIChatRequestPolicy(
|
| 26 |
provider_name="OPENROUTER",
|
|
|
|
| 27 |
include_extra_body=True,
|
| 28 |
extra_body_validator=validate_extra_body_does_not_override_canonical_fields,
|
| 29 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 30 |
)
|
| 31 |
-
_PROFILE = OpenAIChatProfile(_REQUEST_POLICY)
|
| 32 |
|
| 33 |
|
| 34 |
class OpenRouterProvider(OpenAIChatProvider):
|
|
@@ -41,22 +43,6 @@ class OpenRouterProvider(OpenAIChatProvider):
|
|
| 41 |
rate_limiter=rate_limiter,
|
| 42 |
)
|
| 43 |
|
| 44 |
-
def _build_request_body(
|
| 45 |
-
self, request: MessagesRequest, thinking_enabled: bool | None = None
|
| 46 |
-
) -> dict:
|
| 47 |
-
effective_thinking_enabled = self._is_thinking_enabled(
|
| 48 |
-
request, thinking_enabled
|
| 49 |
-
)
|
| 50 |
-
return build_openai_chat_request_body(
|
| 51 |
-
request,
|
| 52 |
-
thinking_enabled=effective_thinking_enabled,
|
| 53 |
-
policy=_REQUEST_POLICY,
|
| 54 |
-
postprocessors=(
|
| 55 |
-
_apply_openrouter_reasoning_policy,
|
| 56 |
-
_apply_openrouter_reasoning_details_replay,
|
| 57 |
-
),
|
| 58 |
-
)
|
| 59 |
-
|
| 60 |
async def list_model_ids(self) -> frozenset[str]:
|
| 61 |
"""Only advertise OpenRouter models that can run Claude Code tools."""
|
| 62 |
payload = await self._client.models.list()
|
|
@@ -72,36 +58,17 @@ class OpenRouterProvider(OpenAIChatProvider):
|
|
| 72 |
)
|
| 73 |
|
| 74 |
def _handle_extra_reasoning(
|
| 75 |
-
self, delta: Any, ledger: AnthropicStreamLedger, *,
|
| 76 |
) -> Iterator[str]:
|
| 77 |
"""Map OpenRouter reasoning details onto Anthropic thinking blocks."""
|
| 78 |
-
if not
|
| 79 |
return iter(())
|
| 80 |
return _iter_openrouter_reasoning_detail_events(delta, ledger)
|
| 81 |
|
| 82 |
|
| 83 |
-
def _apply_openrouter_reasoning_policy(
|
| 84 |
-
body: dict[str, Any], request: MessagesRequest, thinking_enabled: bool
|
| 85 |
-
) -> None:
|
| 86 |
-
if not thinking_enabled:
|
| 87 |
-
return
|
| 88 |
-
extra_body = body.setdefault("extra_body", {})
|
| 89 |
-
if not isinstance(extra_body, dict):
|
| 90 |
-
return
|
| 91 |
-
reasoning = extra_body.setdefault("reasoning", {"enabled": True})
|
| 92 |
-
if not isinstance(reasoning, dict):
|
| 93 |
-
return
|
| 94 |
-
reasoning.setdefault("enabled", True)
|
| 95 |
-
budget_tokens = _thinking_budget_tokens(request.thinking)
|
| 96 |
-
if isinstance(budget_tokens, int):
|
| 97 |
-
reasoning.setdefault("max_tokens", budget_tokens)
|
| 98 |
-
|
| 99 |
-
|
| 100 |
def _apply_openrouter_reasoning_details_replay(
|
| 101 |
-
body: dict[str, Any], request: MessagesRequest,
|
| 102 |
) -> None:
|
| 103 |
-
if not thinking_enabled:
|
| 104 |
-
return
|
| 105 |
assistant_details = _assistant_reasoning_details(request.messages)
|
| 106 |
if not assistant_details:
|
| 107 |
return
|
|
@@ -124,6 +91,13 @@ def _apply_openrouter_reasoning_details_replay(
|
|
| 124 |
break
|
| 125 |
|
| 126 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 127 |
def _assistant_reasoning_details(messages: Any) -> list[list[dict[str, Any]]]:
|
| 128 |
if not _is_sequence(messages):
|
| 129 |
return []
|
|
@@ -157,11 +131,6 @@ def _redacted_reasoning_details(content: Any) -> list[dict[str, Any]]:
|
|
| 157 |
return details
|
| 158 |
|
| 159 |
|
| 160 |
-
def _thinking_budget_tokens(thinking: ThinkingConfig | None) -> int | None:
|
| 161 |
-
value = thinking.budget_tokens if thinking is not None else None
|
| 162 |
-
return value if isinstance(value, int) and not isinstance(value, bool) else None
|
| 163 |
-
|
| 164 |
-
|
| 165 |
def _iter_openrouter_reasoning_detail_events(
|
| 166 |
delta: Any, ledger: AnthropicStreamLedger
|
| 167 |
) -> Iterator[str]:
|
|
|
|
| 6 |
|
| 7 |
from free_claude_code.application.model_metadata import ProviderModelInfo
|
| 8 |
from free_claude_code.config.constants import ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS
|
| 9 |
+
from free_claude_code.core.anthropic import ReasoningReplayMode
|
| 10 |
+
from free_claude_code.core.anthropic.models import MessagesRequest
|
| 11 |
from free_claude_code.core.anthropic.streaming import AnthropicStreamLedger
|
| 12 |
+
from free_claude_code.core.reasoning import ReasoningEffort, ReasoningPolicy
|
| 13 |
from free_claude_code.providers.base import ProviderConfig
|
| 14 |
from free_claude_code.providers.model_listing import (
|
| 15 |
extract_openrouter_tool_model_ids,
|
|
|
|
| 19 |
OpenAIChatProfile,
|
| 20 |
OpenAIChatProvider,
|
| 21 |
OpenAIChatRequestPolicy,
|
| 22 |
+
ReasoningObject,
|
| 23 |
validate_extra_body_does_not_override_canonical_fields,
|
| 24 |
)
|
| 25 |
from free_claude_code.providers.rate_limit import ProviderRateLimiter
|
| 26 |
|
| 27 |
_REQUEST_POLICY = OpenAIChatRequestPolicy(
|
| 28 |
provider_name="OPENROUTER",
|
| 29 |
+
reasoning_replay=ReasoningReplayMode.REASONING_CONTENT,
|
| 30 |
include_extra_body=True,
|
| 31 |
extra_body_validator=validate_extra_body_does_not_override_canonical_fields,
|
| 32 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 33 |
)
|
|
|
|
| 34 |
|
| 35 |
|
| 36 |
class OpenRouterProvider(OpenAIChatProvider):
|
|
|
|
| 43 |
rate_limiter=rate_limiter,
|
| 44 |
)
|
| 45 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
async def list_model_ids(self) -> frozenset[str]:
|
| 47 |
"""Only advertise OpenRouter models that can run Claude Code tools."""
|
| 48 |
payload = await self._client.models.list()
|
|
|
|
| 58 |
)
|
| 59 |
|
| 60 |
def _handle_extra_reasoning(
|
| 61 |
+
self, delta: Any, ledger: AnthropicStreamLedger, *, output_reasoning: bool
|
| 62 |
) -> Iterator[str]:
|
| 63 |
"""Map OpenRouter reasoning details onto Anthropic thinking blocks."""
|
| 64 |
+
if not output_reasoning:
|
| 65 |
return iter(())
|
| 66 |
return _iter_openrouter_reasoning_detail_events(delta, ledger)
|
| 67 |
|
| 68 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
def _apply_openrouter_reasoning_details_replay(
|
| 70 |
+
body: dict[str, Any], request: MessagesRequest, _policy: ReasoningPolicy
|
| 71 |
) -> None:
|
|
|
|
|
|
|
| 72 |
assistant_details = _assistant_reasoning_details(request.messages)
|
| 73 |
if not assistant_details:
|
| 74 |
return
|
|
|
|
| 91 |
break
|
| 92 |
|
| 93 |
|
| 94 |
+
_PROFILE = OpenAIChatProfile(
|
| 95 |
+
_REQUEST_POLICY,
|
| 96 |
+
ReasoningObject(tuple((effort, effort.value) for effort in ReasoningEffort)),
|
| 97 |
+
postprocessors=(_apply_openrouter_reasoning_details_replay,),
|
| 98 |
+
)
|
| 99 |
+
|
| 100 |
+
|
| 101 |
def _assistant_reasoning_details(messages: Any) -> list[list[dict[str, Any]]]:
|
| 102 |
if not _is_sequence(messages):
|
| 103 |
return []
|
|
|
|
| 131 |
return details
|
| 132 |
|
| 133 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 134 |
def _iter_openrouter_reasoning_detail_events(
|
| 135 |
delta: Any, ledger: AnthropicStreamLedger
|
| 136 |
) -> Iterator[str]:
|
|
@@ -7,6 +7,12 @@ from .base_url import openai_v1_base_url
|
|
| 7 |
from .extra_body import validate_extra_body_does_not_override_canonical_fields
|
| 8 |
from .profiles import OPENAI_CHAT_PROFILES, OpenAIChatProfile
|
| 9 |
from .provider import OpenAIChatProvider
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
from .request_policy import OpenAIChatRequestPolicy, build_openai_chat_request_body
|
| 11 |
from .usage import usage_int
|
| 12 |
|
|
@@ -28,10 +34,14 @@ def create_openai_chat_provider(
|
|
| 28 |
|
| 29 |
|
| 30 |
__all__ = [
|
|
|
|
| 31 |
"OPENAI_CHAT_PROFILES",
|
|
|
|
|
|
|
| 32 |
"OpenAIChatProfile",
|
| 33 |
"OpenAIChatProvider",
|
| 34 |
"OpenAIChatRequestPolicy",
|
|
|
|
| 35 |
"build_openai_chat_request_body",
|
| 36 |
"create_openai_chat_provider",
|
| 37 |
"openai_v1_base_url",
|
|
|
|
| 7 |
from .extra_body import validate_extra_body_does_not_override_canonical_fields
|
| 8 |
from .profiles import OPENAI_CHAT_PROFILES, OpenAIChatProfile
|
| 9 |
from .provider import OpenAIChatProvider
|
| 10 |
+
from .reasoning import (
|
| 11 |
+
NO_REASONING,
|
| 12 |
+
ChatTemplateReasoning,
|
| 13 |
+
NamedEffortReasoning,
|
| 14 |
+
ReasoningObject,
|
| 15 |
+
)
|
| 16 |
from .request_policy import OpenAIChatRequestPolicy, build_openai_chat_request_body
|
| 17 |
from .usage import usage_int
|
| 18 |
|
|
|
|
| 34 |
|
| 35 |
|
| 36 |
__all__ = [
|
| 37 |
+
"NO_REASONING",
|
| 38 |
"OPENAI_CHAT_PROFILES",
|
| 39 |
+
"ChatTemplateReasoning",
|
| 40 |
+
"NamedEffortReasoning",
|
| 41 |
"OpenAIChatProfile",
|
| 42 |
"OpenAIChatProvider",
|
| 43 |
"OpenAIChatRequestPolicy",
|
| 44 |
+
"ReasoningObject",
|
| 45 |
"build_openai_chat_request_body",
|
| 46 |
"create_openai_chat_provider",
|
| 47 |
"openai_v1_base_url",
|
|
@@ -17,6 +17,21 @@ CANONICAL_OPENAI_CHAT_BODY_KEYS = frozenset(
|
|
| 17 |
"stop",
|
| 18 |
"stop_sequences",
|
| 19 |
"stream_options",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
}
|
| 21 |
)
|
| 22 |
|
|
@@ -30,3 +45,15 @@ def validate_extra_body_does_not_override_canonical_fields(
|
|
| 30 |
raise ValueError(
|
| 31 |
f"extra_body must not override canonical request fields: {sorted(bad)}"
|
| 32 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
"stop",
|
| 18 |
"stop_sequences",
|
| 19 |
"stream_options",
|
| 20 |
+
"reasoning",
|
| 21 |
+
"reasoning_effort",
|
| 22 |
+
"reasoning_tokens",
|
| 23 |
+
"thinking",
|
| 24 |
+
"thinking_budget_tokens",
|
| 25 |
+
}
|
| 26 |
+
)
|
| 27 |
+
|
| 28 |
+
REASONING_OPENAI_CHAT_BODY_KEYS = frozenset(
|
| 29 |
+
{
|
| 30 |
+
"reasoning",
|
| 31 |
+
"reasoning_effort",
|
| 32 |
+
"reasoning_tokens",
|
| 33 |
+
"thinking",
|
| 34 |
+
"thinking_budget_tokens",
|
| 35 |
}
|
| 36 |
)
|
| 37 |
|
|
|
|
| 45 |
raise ValueError(
|
| 46 |
f"extra_body must not override canonical request fields: {sorted(bad)}"
|
| 47 |
)
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def validate_extra_body_does_not_override_reasoning_fields(
|
| 51 |
+
extra: dict[str, Any],
|
| 52 |
+
) -> None:
|
| 53 |
+
"""Keep provider reasoning translation authoritative over caller extras."""
|
| 54 |
+
|
| 55 |
+
bad = REASONING_OPENAI_CHAT_BODY_KEYS & extra.keys()
|
| 56 |
+
if bad:
|
| 57 |
+
raise ValueError(
|
| 58 |
+
f"extra_body must not override reasoning fields: {sorted(bad)}"
|
| 59 |
+
)
|
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
"""Declarative profiles for
|
| 2 |
|
| 3 |
from collections.abc import Mapping
|
| 4 |
from copy import deepcopy
|
|
@@ -9,17 +9,49 @@ from free_claude_code.application.errors import InvalidRequestError
|
|
| 9 |
from free_claude_code.config.constants import ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS
|
| 10 |
from free_claude_code.core.anthropic import ReasoningReplayMode
|
| 11 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
|
|
|
| 12 |
|
| 13 |
from .base_url import openai_v1_base_url
|
| 14 |
-
from .extra_body import
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
from .request_policy import OpenAIChatPostprocessor, OpenAIChatRequestPolicy
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
@dataclass(frozen=True, slots=True)
|
| 19 |
class OpenAIChatProfile:
|
| 20 |
-
"""Immutable
|
| 21 |
|
| 22 |
request_policy: OpenAIChatRequestPolicy
|
|
|
|
| 23 |
postprocessors: tuple[OpenAIChatPostprocessor, ...] = ()
|
| 24 |
normalize_base_url: bool = False
|
| 25 |
reasoning_delta_field: Literal["reasoning_content", "reasoning"] = (
|
|
@@ -37,12 +69,23 @@ class OpenAIChatProfile:
|
|
| 37 |
value = getattr(delta, self.reasoning_delta_field, None)
|
| 38 |
return value if isinstance(value, str) else None
|
| 39 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
def _apply_cohere_request_quirks(
|
| 42 |
-
body: dict[str, Any], request: MessagesRequest,
|
| 43 |
) -> None:
|
| 44 |
_merge_allowed_cohere_extra_body(body, request.extra_body)
|
| 45 |
-
body["reasoning_effort"] = "high" if thinking_enabled else "none"
|
| 46 |
|
| 47 |
|
| 48 |
_COHERE_EXTRA_BODY_KEYS = frozenset(
|
|
@@ -72,78 +115,53 @@ def _merge_allowed_cohere_extra_body(body: dict[str, Any], extra_body: Any) -> N
|
|
| 72 |
body.update({str(key): deepcopy(value) for key, value in extra_body.items()})
|
| 73 |
|
| 74 |
|
| 75 |
-
def
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
def _apply_minimax_thinking_policy(
|
| 86 |
-
body: dict[str, Any], _request: MessagesRequest, thinking_enabled: bool
|
| 87 |
-
) -> None:
|
| 88 |
-
extra_body = body.setdefault("extra_body", {})
|
| 89 |
-
if not isinstance(extra_body, dict):
|
| 90 |
-
return
|
| 91 |
-
extra_body["reasoning_split"] = True
|
| 92 |
-
extra_body["thinking"] = (
|
| 93 |
-
{"type": "adaptive"} if thinking_enabled else {"type": "disabled"}
|
| 94 |
-
)
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
def _apply_ollama_thinking_policy(
|
| 98 |
-
body: dict[str, Any], _request: MessagesRequest, thinking_enabled: bool
|
| 99 |
-
) -> None:
|
| 100 |
-
body["reasoning_effort"] = "high" if thinking_enabled else "none"
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
def _apply_wafer_thinking_policy(
|
| 104 |
-
body: dict[str, Any], _request: MessagesRequest, thinking_enabled: bool
|
| 105 |
-
) -> None:
|
| 106 |
-
extra_body = body.setdefault("extra_body", {})
|
| 107 |
-
if isinstance(extra_body, dict):
|
| 108 |
-
extra_body["thinking"] = (
|
| 109 |
-
{"type": "enabled"} if thinking_enabled else {"type": "disabled"}
|
| 110 |
-
)
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
def _apply_zai_thinking_policy(
|
| 114 |
-
body: dict[str, Any], _request: MessagesRequest, thinking_enabled: bool
|
| 115 |
-
) -> None:
|
| 116 |
-
extra_body = body.setdefault("extra_body", {})
|
| 117 |
-
if not isinstance(extra_body, dict):
|
| 118 |
-
return
|
| 119 |
-
extra_body["thinking"] = (
|
| 120 |
-
{"type": "enabled", "clear_thinking": False}
|
| 121 |
-
if thinking_enabled
|
| 122 |
-
else {"type": "disabled"}
|
| 123 |
)
|
| 124 |
|
| 125 |
|
| 126 |
OPENAI_CHAT_PROFILES: dict[str, OpenAIChatProfile] = {
|
| 127 |
"mistral_codestral": OpenAIChatProfile(
|
| 128 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 129 |
),
|
| 130 |
-
"opencode": OpenAIChatProfile(OpenAIChatRequestPolicy(provider_name="OPENCODE")),
|
| 131 |
"opencode_go": OpenAIChatProfile(
|
| 132 |
-
|
|
|
|
| 133 |
),
|
| 134 |
"vercel": OpenAIChatProfile(
|
| 135 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 136 |
),
|
| 137 |
"huggingface": OpenAIChatProfile(
|
| 138 |
-
|
| 139 |
-
|
|
|
|
| 140 |
include_extra_body=True,
|
| 141 |
-
|
| 142 |
-
)
|
|
|
|
| 143 |
),
|
| 144 |
"cohere": OpenAIChatProfile(
|
| 145 |
-
|
| 146 |
-
|
|
|
|
| 147 |
strip_message_names=True,
|
| 148 |
unsupported_body_keys=frozenset(
|
| 149 |
{
|
|
@@ -160,94 +178,162 @@ OPENAI_CHAT_PROFILES: dict[str, OpenAIChatProfile] = {
|
|
| 160 |
}
|
| 161 |
),
|
| 162 |
),
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 163 |
postprocessors=(_apply_cohere_request_quirks,),
|
| 164 |
),
|
| 165 |
"wafer": OpenAIChatProfile(
|
| 166 |
-
|
| 167 |
-
|
|
|
|
| 168 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 169 |
),
|
| 170 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 171 |
),
|
| 172 |
"kimi": OpenAIChatProfile(
|
| 173 |
-
|
| 174 |
-
|
|
|
|
| 175 |
reject_extra_body_message=(
|
| 176 |
"Kimi Chat Completions API does not support caller extra_body on requests."
|
| 177 |
),
|
| 178 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 179 |
),
|
| 180 |
-
|
|
|
|
|
|
|
|
|
|
| 181 |
),
|
| 182 |
"minimax": OpenAIChatProfile(
|
| 183 |
-
|
| 184 |
-
|
|
|
|
| 185 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 186 |
max_tokens_field="max_completion_tokens",
|
| 187 |
),
|
| 188 |
-
|
| 189 |
),
|
| 190 |
"cerebras": OpenAIChatProfile(
|
| 191 |
-
|
| 192 |
-
|
|
|
|
| 193 |
include_extra_body=True,
|
|
|
|
| 194 |
max_tokens_field="max_completion_tokens",
|
| 195 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 196 |
),
|
| 197 |
reasoning_delta_field="reasoning",
|
| 198 |
),
|
| 199 |
"groq": OpenAIChatProfile(
|
| 200 |
-
|
| 201 |
-
|
|
|
|
| 202 |
include_extra_body=True,
|
|
|
|
| 203 |
max_tokens_field="max_completion_tokens",
|
| 204 |
strip_message_names=True,
|
| 205 |
unsupported_body_keys=frozenset({"logprobs", "logit_bias", "top_logprobs"}),
|
| 206 |
normalize_n_to_one=True,
|
| 207 |
-
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 208 |
),
|
| 209 |
"sambanova": OpenAIChatProfile(
|
| 210 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 211 |
),
|
| 212 |
"fireworks": OpenAIChatProfile(
|
| 213 |
-
|
| 214 |
-
|
|
|
|
| 215 |
include_extra_body=True,
|
| 216 |
extra_body_validator=validate_extra_body_does_not_override_canonical_fields,
|
| 217 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 218 |
-
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 219 |
),
|
| 220 |
"zai": OpenAIChatProfile(
|
| 221 |
-
|
| 222 |
-
|
|
|
|
| 223 |
reject_extra_body_message=(
|
| 224 |
"Z.ai Chat Completions API does not support caller extra_body on requests."
|
| 225 |
),
|
| 226 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 227 |
),
|
| 228 |
-
|
|
|
|
|
|
|
|
|
|
| 229 |
),
|
| 230 |
"ollama_cloud": OpenAIChatProfile(
|
| 231 |
-
|
| 232 |
-
|
|
|
|
| 233 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 234 |
-
reasoning_replay=ReasoningReplayMode.REASONING,
|
| 235 |
),
|
| 236 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 237 |
reasoning_delta_field="reasoning",
|
| 238 |
),
|
| 239 |
"llamacpp": OpenAIChatProfile(
|
| 240 |
-
|
| 241 |
-
|
|
|
|
| 242 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 243 |
),
|
|
|
|
| 244 |
normalize_base_url=True,
|
| 245 |
),
|
| 246 |
"ollama": OpenAIChatProfile(
|
| 247 |
-
|
| 248 |
-
|
|
|
|
| 249 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 250 |
),
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 251 |
normalize_base_url=True,
|
|
|
|
| 252 |
),
|
| 253 |
}
|
|
|
|
| 1 |
+
"""Declarative profiles for ordinary OpenAI-compatible providers."""
|
| 2 |
|
| 3 |
from collections.abc import Mapping
|
| 4 |
from copy import deepcopy
|
|
|
|
| 9 |
from free_claude_code.config.constants import ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS
|
| 10 |
from free_claude_code.core.anthropic import ReasoningReplayMode
|
| 11 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
| 12 |
+
from free_claude_code.core.reasoning import ReasoningEffort, ReasoningPolicy
|
| 13 |
|
| 14 |
from .base_url import openai_v1_base_url
|
| 15 |
+
from .extra_body import (
|
| 16 |
+
validate_extra_body_does_not_override_canonical_fields,
|
| 17 |
+
validate_extra_body_does_not_override_reasoning_fields,
|
| 18 |
+
)
|
| 19 |
+
from .reasoning import (
|
| 20 |
+
LLAMACPP_REASONING,
|
| 21 |
+
NO_REASONING,
|
| 22 |
+
SPLIT_REASONING_OUTPUT,
|
| 23 |
+
NamedEffortReasoning,
|
| 24 |
+
ReasoningEncoder,
|
| 25 |
+
ReasoningObject,
|
| 26 |
+
ThinkingObjectReasoning,
|
| 27 |
+
)
|
| 28 |
from .request_policy import OpenAIChatPostprocessor, OpenAIChatRequestPolicy
|
| 29 |
|
| 30 |
+
_ALL_EFFORTS = tuple((effort, effort.value) for effort in ReasoningEffort)
|
| 31 |
+
_LOW_MEDIUM_HIGH = (
|
| 32 |
+
(ReasoningEffort.MINIMAL, "low"),
|
| 33 |
+
(ReasoningEffort.LOW, "low"),
|
| 34 |
+
(ReasoningEffort.MEDIUM, "medium"),
|
| 35 |
+
(ReasoningEffort.HIGH, "high"),
|
| 36 |
+
(ReasoningEffort.XHIGH, "high"),
|
| 37 |
+
(ReasoningEffort.MAX, "high"),
|
| 38 |
+
)
|
| 39 |
+
_LOW_TO_MAX = (
|
| 40 |
+
(ReasoningEffort.MINIMAL, "low"),
|
| 41 |
+
(ReasoningEffort.LOW, "low"),
|
| 42 |
+
(ReasoningEffort.MEDIUM, "medium"),
|
| 43 |
+
(ReasoningEffort.HIGH, "high"),
|
| 44 |
+
(ReasoningEffort.XHIGH, "max"),
|
| 45 |
+
(ReasoningEffort.MAX, "max"),
|
| 46 |
+
)
|
| 47 |
+
|
| 48 |
|
| 49 |
@dataclass(frozen=True, slots=True)
|
| 50 |
class OpenAIChatProfile:
|
| 51 |
+
"""Immutable transport and reasoning behavior for one provider."""
|
| 52 |
|
| 53 |
request_policy: OpenAIChatRequestPolicy
|
| 54 |
+
reasoning: ReasoningEncoder
|
| 55 |
postprocessors: tuple[OpenAIChatPostprocessor, ...] = ()
|
| 56 |
normalize_base_url: bool = False
|
| 57 |
reasoning_delta_field: Literal["reasoning_content", "reasoning"] = (
|
|
|
|
| 69 |
value = getattr(delta, self.reasoning_delta_field, None)
|
| 70 |
return value if isinstance(value, str) else None
|
| 71 |
|
| 72 |
+
def apply_reasoning(
|
| 73 |
+
self,
|
| 74 |
+
body: dict[str, Any],
|
| 75 |
+
_request: MessagesRequest,
|
| 76 |
+
policy: ReasoningPolicy,
|
| 77 |
+
) -> None:
|
| 78 |
+
self.reasoning.encode(body, policy)
|
| 79 |
+
|
| 80 |
+
@property
|
| 81 |
+
def request_postprocessors(self) -> tuple[OpenAIChatPostprocessor, ...]:
|
| 82 |
+
return (*self.postprocessors, self.apply_reasoning)
|
| 83 |
+
|
| 84 |
|
| 85 |
def _apply_cohere_request_quirks(
|
| 86 |
+
body: dict[str, Any], request: MessagesRequest, _policy: ReasoningPolicy
|
| 87 |
) -> None:
|
| 88 |
_merge_allowed_cohere_extra_body(body, request.extra_body)
|
|
|
|
| 89 |
|
| 90 |
|
| 91 |
_COHERE_EXTRA_BODY_KEYS = frozenset(
|
|
|
|
| 115 |
body.update({str(key): deepcopy(value) for key, value in extra_body.items()})
|
| 116 |
|
| 117 |
|
| 118 |
+
def _policy(
|
| 119 |
+
provider_name: str,
|
| 120 |
+
replay: ReasoningReplayMode,
|
| 121 |
+
**kwargs: Any,
|
| 122 |
+
) -> OpenAIChatRequestPolicy:
|
| 123 |
+
return OpenAIChatRequestPolicy(
|
| 124 |
+
provider_name=provider_name,
|
| 125 |
+
reasoning_replay=replay,
|
| 126 |
+
**kwargs,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 127 |
)
|
| 128 |
|
| 129 |
|
| 130 |
OPENAI_CHAT_PROFILES: dict[str, OpenAIChatProfile] = {
|
| 131 |
"mistral_codestral": OpenAIChatProfile(
|
| 132 |
+
_policy("CODESTRAL", ReasoningReplayMode.THINK_TAGS),
|
| 133 |
+
NO_REASONING,
|
| 134 |
+
),
|
| 135 |
+
"opencode": OpenAIChatProfile(
|
| 136 |
+
_policy("OPENCODE", ReasoningReplayMode.THINK_TAGS),
|
| 137 |
+
NO_REASONING,
|
| 138 |
),
|
|
|
|
| 139 |
"opencode_go": OpenAIChatProfile(
|
| 140 |
+
_policy("OPENCODE_GO", ReasoningReplayMode.THINK_TAGS),
|
| 141 |
+
NO_REASONING,
|
| 142 |
),
|
| 143 |
"vercel": OpenAIChatProfile(
|
| 144 |
+
_policy(
|
| 145 |
+
"VERCEL",
|
| 146 |
+
ReasoningReplayMode.THINK_TAGS,
|
| 147 |
+
include_extra_body=True,
|
| 148 |
+
extra_body_validator=validate_extra_body_does_not_override_reasoning_fields,
|
| 149 |
+
),
|
| 150 |
+
ReasoningObject(_ALL_EFFORTS),
|
| 151 |
),
|
| 152 |
"huggingface": OpenAIChatProfile(
|
| 153 |
+
_policy(
|
| 154 |
+
"HUGGINGFACE",
|
| 155 |
+
ReasoningReplayMode.DISABLED,
|
| 156 |
include_extra_body=True,
|
| 157 |
+
extra_body_validator=validate_extra_body_does_not_override_reasoning_fields,
|
| 158 |
+
),
|
| 159 |
+
NO_REASONING,
|
| 160 |
),
|
| 161 |
"cohere": OpenAIChatProfile(
|
| 162 |
+
_policy(
|
| 163 |
+
"COHERE",
|
| 164 |
+
ReasoningReplayMode.REASONING_CONTENT,
|
| 165 |
strip_message_names=True,
|
| 166 |
unsupported_body_keys=frozenset(
|
| 167 |
{
|
|
|
|
| 178 |
}
|
| 179 |
),
|
| 180 |
),
|
| 181 |
+
NamedEffortReasoning(
|
| 182 |
+
tuple((effort, "high") for effort in ReasoningEffort),
|
| 183 |
+
disabled_value="none",
|
| 184 |
+
enabled_value="high",
|
| 185 |
+
),
|
| 186 |
postprocessors=(_apply_cohere_request_quirks,),
|
| 187 |
),
|
| 188 |
"wafer": OpenAIChatProfile(
|
| 189 |
+
_policy(
|
| 190 |
+
"WAFER",
|
| 191 |
+
ReasoningReplayMode.REASONING_CONTENT,
|
| 192 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 193 |
),
|
| 194 |
+
NamedEffortReasoning(
|
| 195 |
+
_LOW_TO_MAX,
|
| 196 |
+
disabled_value="none",
|
| 197 |
+
enabled_value="high",
|
| 198 |
+
),
|
| 199 |
),
|
| 200 |
"kimi": OpenAIChatProfile(
|
| 201 |
+
_policy(
|
| 202 |
+
"KIMI",
|
| 203 |
+
ReasoningReplayMode.REASONING_CONTENT,
|
| 204 |
reject_extra_body_message=(
|
| 205 |
"Kimi Chat Completions API does not support caller extra_body on requests."
|
| 206 |
),
|
| 207 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 208 |
),
|
| 209 |
+
ThinkingObjectReasoning(
|
| 210 |
+
enabled={"type": "enabled"},
|
| 211 |
+
disabled={"type": "disabled"},
|
| 212 |
+
),
|
| 213 |
),
|
| 214 |
"minimax": OpenAIChatProfile(
|
| 215 |
+
_policy(
|
| 216 |
+
"MINIMAX",
|
| 217 |
+
ReasoningReplayMode.REASONING_CONTENT,
|
| 218 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 219 |
max_tokens_field="max_completion_tokens",
|
| 220 |
),
|
| 221 |
+
SPLIT_REASONING_OUTPUT,
|
| 222 |
),
|
| 223 |
"cerebras": OpenAIChatProfile(
|
| 224 |
+
_policy(
|
| 225 |
+
"CEREBRAS",
|
| 226 |
+
ReasoningReplayMode.THINK_TAGS,
|
| 227 |
include_extra_body=True,
|
| 228 |
+
extra_body_validator=validate_extra_body_does_not_override_reasoning_fields,
|
| 229 |
max_tokens_field="max_completion_tokens",
|
| 230 |
+
),
|
| 231 |
+
NamedEffortReasoning(
|
| 232 |
+
_LOW_MEDIUM_HIGH,
|
| 233 |
+
disabled_value="none",
|
| 234 |
+
enabled_value="medium",
|
| 235 |
),
|
| 236 |
reasoning_delta_field="reasoning",
|
| 237 |
),
|
| 238 |
"groq": OpenAIChatProfile(
|
| 239 |
+
_policy(
|
| 240 |
+
"GROQ",
|
| 241 |
+
ReasoningReplayMode.REASONING_CONTENT,
|
| 242 |
include_extra_body=True,
|
| 243 |
+
extra_body_validator=validate_extra_body_does_not_override_reasoning_fields,
|
| 244 |
max_tokens_field="max_completion_tokens",
|
| 245 |
strip_message_names=True,
|
| 246 |
unsupported_body_keys=frozenset({"logprobs", "logit_bias", "top_logprobs"}),
|
| 247 |
normalize_n_to_one=True,
|
| 248 |
+
),
|
| 249 |
+
NamedEffortReasoning(
|
| 250 |
+
_LOW_MEDIUM_HIGH,
|
| 251 |
+
disabled_value="none",
|
| 252 |
+
enabled_value="medium",
|
| 253 |
+
),
|
| 254 |
),
|
| 255 |
"sambanova": OpenAIChatProfile(
|
| 256 |
+
_policy(
|
| 257 |
+
"SAMBANOVA",
|
| 258 |
+
ReasoningReplayMode.REASONING_CONTENT,
|
| 259 |
+
include_extra_body=True,
|
| 260 |
+
extra_body_validator=validate_extra_body_does_not_override_reasoning_fields,
|
| 261 |
+
),
|
| 262 |
+
NamedEffortReasoning(
|
| 263 |
+
_LOW_MEDIUM_HIGH,
|
| 264 |
+
enabled_value="medium",
|
| 265 |
+
),
|
| 266 |
),
|
| 267 |
"fireworks": OpenAIChatProfile(
|
| 268 |
+
_policy(
|
| 269 |
+
"FIREWORKS",
|
| 270 |
+
ReasoningReplayMode.REASONING_CONTENT,
|
| 271 |
include_extra_body=True,
|
| 272 |
extra_body_validator=validate_extra_body_does_not_override_canonical_fields,
|
| 273 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 274 |
+
),
|
| 275 |
+
NamedEffortReasoning(
|
| 276 |
+
(
|
| 277 |
+
(ReasoningEffort.MINIMAL, "low"),
|
| 278 |
+
(ReasoningEffort.LOW, "low"),
|
| 279 |
+
(ReasoningEffort.MEDIUM, "medium"),
|
| 280 |
+
(ReasoningEffort.HIGH, "high"),
|
| 281 |
+
(ReasoningEffort.XHIGH, "xhigh"),
|
| 282 |
+
(ReasoningEffort.MAX, "max"),
|
| 283 |
+
),
|
| 284 |
+
disabled_value="none",
|
| 285 |
+
enabled_value="high",
|
| 286 |
+
budget_field="reasoning_effort",
|
| 287 |
+
),
|
| 288 |
),
|
| 289 |
"zai": OpenAIChatProfile(
|
| 290 |
+
_policy(
|
| 291 |
+
"ZAI",
|
| 292 |
+
ReasoningReplayMode.REASONING_CONTENT,
|
| 293 |
reject_extra_body_message=(
|
| 294 |
"Z.ai Chat Completions API does not support caller extra_body on requests."
|
| 295 |
),
|
| 296 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 297 |
),
|
| 298 |
+
ThinkingObjectReasoning(
|
| 299 |
+
enabled={"type": "enabled", "clear_thinking": False},
|
| 300 |
+
disabled={"type": "disabled"},
|
| 301 |
+
),
|
| 302 |
),
|
| 303 |
"ollama_cloud": OpenAIChatProfile(
|
| 304 |
+
_policy(
|
| 305 |
+
"OLLAMA_CLOUD",
|
| 306 |
+
ReasoningReplayMode.REASONING,
|
| 307 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
|
|
|
| 308 |
),
|
| 309 |
+
NamedEffortReasoning(
|
| 310 |
+
_LOW_TO_MAX,
|
| 311 |
+
disabled_value="none",
|
| 312 |
+
enabled_value="high",
|
| 313 |
+
),
|
| 314 |
reasoning_delta_field="reasoning",
|
| 315 |
),
|
| 316 |
"llamacpp": OpenAIChatProfile(
|
| 317 |
+
_policy(
|
| 318 |
+
"LLAMACPP",
|
| 319 |
+
ReasoningReplayMode.THINK_TAGS,
|
| 320 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 321 |
),
|
| 322 |
+
LLAMACPP_REASONING,
|
| 323 |
normalize_base_url=True,
|
| 324 |
),
|
| 325 |
"ollama": OpenAIChatProfile(
|
| 326 |
+
_policy(
|
| 327 |
+
"OLLAMA",
|
| 328 |
+
ReasoningReplayMode.REASONING,
|
| 329 |
default_max_tokens=ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
|
| 330 |
),
|
| 331 |
+
NamedEffortReasoning(
|
| 332 |
+
_LOW_TO_MAX,
|
| 333 |
+
disabled_value="none",
|
| 334 |
+
enabled_value="high",
|
| 335 |
+
),
|
| 336 |
normalize_base_url=True,
|
| 337 |
+
reasoning_delta_field="reasoning",
|
| 338 |
),
|
| 339 |
}
|
|
@@ -27,6 +27,7 @@ from free_claude_code.core.anthropic.streaming import (
|
|
| 27 |
tool_schemas_by_name,
|
| 28 |
)
|
| 29 |
from free_claude_code.core.failures import ExecutionFailure
|
|
|
|
| 30 |
from free_claude_code.core.trace import provider_chat_body_snapshot, trace_event
|
| 31 |
from free_claude_code.providers.base import BaseProvider, ProviderConfig
|
| 32 |
from free_claude_code.providers.failure_policy import classify_provider_failure
|
|
@@ -120,24 +121,30 @@ class OpenAIChatProvider(BaseProvider):
|
|
| 120 |
return extract_openai_model_ids(payload, provider_name=self._provider_name)
|
| 121 |
|
| 122 |
def _build_request_body(
|
| 123 |
-
self,
|
|
|
|
|
|
|
|
|
|
| 124 |
) -> dict[str, Any]:
|
| 125 |
"""Build a provider request from the immutable profile."""
|
| 126 |
return build_openai_chat_request_body(
|
| 127 |
request,
|
| 128 |
-
|
| 129 |
policy=self._profile.request_policy,
|
| 130 |
-
postprocessors=self._profile.
|
| 131 |
)
|
| 132 |
|
| 133 |
def preflight_stream(
|
| 134 |
-
self,
|
|
|
|
|
|
|
|
|
|
| 135 |
) -> None:
|
| 136 |
"""Validate OpenAI-chat request conversion before streaming."""
|
| 137 |
-
self._build_request_body(request,
|
| 138 |
|
| 139 |
def _handle_extra_reasoning(
|
| 140 |
-
self, delta: Any, ledger: AnthropicStreamLedger, *,
|
| 141 |
) -> Iterator[str]:
|
| 142 |
"""Hook for provider-specific reasoning."""
|
| 143 |
return iter(())
|
|
@@ -254,7 +261,7 @@ class OpenAIChatProvider(BaseProvider):
|
|
| 254 |
input_tokens: int = 0,
|
| 255 |
*,
|
| 256 |
request_id: str | None = None,
|
| 257 |
-
|
| 258 |
) -> AsyncIterator[str]:
|
| 259 |
"""Stream response in Anthropic SSE format."""
|
| 260 |
runner = _OpenAIChatStreamRunner(
|
|
@@ -262,7 +269,7 @@ class OpenAIChatProvider(BaseProvider):
|
|
| 262 |
request=request,
|
| 263 |
input_tokens=input_tokens,
|
| 264 |
request_id=request_id,
|
| 265 |
-
|
| 266 |
)
|
| 267 |
return runner.run()
|
| 268 |
|
|
@@ -277,13 +284,13 @@ class _OpenAIChatStreamRunner:
|
|
| 277 |
request: MessagesRequest,
|
| 278 |
input_tokens: int,
|
| 279 |
request_id: str | None,
|
| 280 |
-
|
| 281 |
) -> None:
|
| 282 |
self._provider = provider
|
| 283 |
self._request = request
|
| 284 |
self._input_tokens = input_tokens
|
| 285 |
self._request_id = request_id
|
| 286 |
-
self.
|
| 287 |
self._message_id = f"msg_{uuid.uuid4()}"
|
| 288 |
self._tool_calls = OpenAIToolCallAssembler(
|
| 289 |
record_extra_content=provider._record_tool_call_extra_content
|
|
@@ -307,12 +314,11 @@ class _OpenAIChatStreamRunner:
|
|
| 307 |
yield from hold_event(event)
|
| 308 |
|
| 309 |
body = self._provider._build_request_body(
|
| 310 |
-
self._request,
|
|
|
|
| 311 |
)
|
| 312 |
request_stream_usage(body)
|
| 313 |
-
|
| 314 |
-
self._request, self._thinking_enabled
|
| 315 |
-
)
|
| 316 |
trace_event(
|
| 317 |
stage="provider",
|
| 318 |
event="provider.request.sent",
|
|
@@ -362,7 +368,7 @@ class _OpenAIChatStreamRunner:
|
|
| 362 |
logger.debug("{} finish_reason: {}", tag, finish_reason)
|
| 363 |
|
| 364 |
reasoning = self._provider._profile.reasoning_delta(delta)
|
| 365 |
-
if
|
| 366 |
for event in hold_events(ledger.ensure_thinking_block()):
|
| 367 |
yield event
|
| 368 |
if reasoning:
|
|
@@ -374,7 +380,7 @@ class _OpenAIChatStreamRunner:
|
|
| 374 |
for event in self._provider._handle_extra_reasoning(
|
| 375 |
delta,
|
| 376 |
ledger,
|
| 377 |
-
|
| 378 |
):
|
| 379 |
for out_event in hold_event(event):
|
| 380 |
yield out_event
|
|
@@ -382,7 +388,7 @@ class _OpenAIChatStreamRunner:
|
|
| 382 |
if delta.content:
|
| 383 |
for part in think_parser.feed(delta.content):
|
| 384 |
if part.type == ContentType.THINKING:
|
| 385 |
-
if not
|
| 386 |
continue
|
| 387 |
for event in hold_events(
|
| 388 |
ledger.ensure_thinking_block()
|
|
@@ -477,7 +483,7 @@ class _OpenAIChatStreamRunner:
|
|
| 477 |
ledger=ledger,
|
| 478 |
error=error,
|
| 479 |
tool_argument_alias_buffers=tool_argument_alias_buffers,
|
| 480 |
-
|
| 481 |
)
|
| 482 |
except Exception as recovery_error:
|
| 483 |
trace_event(
|
|
@@ -550,7 +556,7 @@ class _OpenAIChatStreamRunner:
|
|
| 550 |
remaining = think_parser.flush()
|
| 551 |
if remaining:
|
| 552 |
if remaining.type == ContentType.THINKING:
|
| 553 |
-
if not
|
| 554 |
remaining = None
|
| 555 |
else:
|
| 556 |
for event in hold_events(ledger.ensure_thinking_block()):
|
|
@@ -701,7 +707,7 @@ class _OpenAIChatStreamRunner:
|
|
| 701 |
ledger: AnthropicStreamLedger,
|
| 702 |
error: Exception,
|
| 703 |
tool_argument_alias_buffers: dict[int, str],
|
| 704 |
-
|
| 705 |
) -> list[str] | None:
|
| 706 |
"""Build terminal recovery events when the interrupted stream permits it."""
|
| 707 |
if not is_retryable_stream_error(error):
|
|
@@ -743,7 +749,7 @@ class _OpenAIChatStreamRunner:
|
|
| 743 |
|
| 744 |
recovery_body = make_text_recovery_body(body, partial_text, partial_thinking)
|
| 745 |
text, thinking = await self._collect_recovery_text(
|
| 746 |
-
recovery_body, include_reasoning=
|
| 747 |
)
|
| 748 |
text_suffix = continuation_suffix(partial_text, text)
|
| 749 |
thinking_suffix = continuation_suffix(partial_thinking, thinking)
|
|
|
|
| 27 |
tool_schemas_by_name,
|
| 28 |
)
|
| 29 |
from free_claude_code.core.failures import ExecutionFailure
|
| 30 |
+
from free_claude_code.core.reasoning import DEFAULT_REASONING_POLICY, ReasoningPolicy
|
| 31 |
from free_claude_code.core.trace import provider_chat_body_snapshot, trace_event
|
| 32 |
from free_claude_code.providers.base import BaseProvider, ProviderConfig
|
| 33 |
from free_claude_code.providers.failure_policy import classify_provider_failure
|
|
|
|
| 121 |
return extract_openai_model_ids(payload, provider_name=self._provider_name)
|
| 122 |
|
| 123 |
def _build_request_body(
|
| 124 |
+
self,
|
| 125 |
+
request: MessagesRequest,
|
| 126 |
+
*,
|
| 127 |
+
reasoning: ReasoningPolicy = DEFAULT_REASONING_POLICY,
|
| 128 |
) -> dict[str, Any]:
|
| 129 |
"""Build a provider request from the immutable profile."""
|
| 130 |
return build_openai_chat_request_body(
|
| 131 |
request,
|
| 132 |
+
reasoning=reasoning,
|
| 133 |
policy=self._profile.request_policy,
|
| 134 |
+
postprocessors=self._profile.request_postprocessors,
|
| 135 |
)
|
| 136 |
|
| 137 |
def preflight_stream(
|
| 138 |
+
self,
|
| 139 |
+
request: MessagesRequest,
|
| 140 |
+
*,
|
| 141 |
+
reasoning: ReasoningPolicy = DEFAULT_REASONING_POLICY,
|
| 142 |
) -> None:
|
| 143 |
"""Validate OpenAI-chat request conversion before streaming."""
|
| 144 |
+
self._build_request_body(request, reasoning=reasoning)
|
| 145 |
|
| 146 |
def _handle_extra_reasoning(
|
| 147 |
+
self, delta: Any, ledger: AnthropicStreamLedger, *, output_reasoning: bool
|
| 148 |
) -> Iterator[str]:
|
| 149 |
"""Hook for provider-specific reasoning."""
|
| 150 |
return iter(())
|
|
|
|
| 261 |
input_tokens: int = 0,
|
| 262 |
*,
|
| 263 |
request_id: str | None = None,
|
| 264 |
+
reasoning: ReasoningPolicy = DEFAULT_REASONING_POLICY,
|
| 265 |
) -> AsyncIterator[str]:
|
| 266 |
"""Stream response in Anthropic SSE format."""
|
| 267 |
runner = _OpenAIChatStreamRunner(
|
|
|
|
| 269 |
request=request,
|
| 270 |
input_tokens=input_tokens,
|
| 271 |
request_id=request_id,
|
| 272 |
+
reasoning=reasoning,
|
| 273 |
)
|
| 274 |
return runner.run()
|
| 275 |
|
|
|
|
| 284 |
request: MessagesRequest,
|
| 285 |
input_tokens: int,
|
| 286 |
request_id: str | None,
|
| 287 |
+
reasoning: ReasoningPolicy,
|
| 288 |
) -> None:
|
| 289 |
self._provider = provider
|
| 290 |
self._request = request
|
| 291 |
self._input_tokens = input_tokens
|
| 292 |
self._request_id = request_id
|
| 293 |
+
self._reasoning = reasoning
|
| 294 |
self._message_id = f"msg_{uuid.uuid4()}"
|
| 295 |
self._tool_calls = OpenAIToolCallAssembler(
|
| 296 |
record_extra_content=provider._record_tool_call_extra_content
|
|
|
|
| 314 |
yield from hold_event(event)
|
| 315 |
|
| 316 |
body = self._provider._build_request_body(
|
| 317 |
+
self._request,
|
| 318 |
+
reasoning=self._reasoning,
|
| 319 |
)
|
| 320 |
request_stream_usage(body)
|
| 321 |
+
output_reasoning = self._reasoning.output_enabled
|
|
|
|
|
|
|
| 322 |
trace_event(
|
| 323 |
stage="provider",
|
| 324 |
event="provider.request.sent",
|
|
|
|
| 368 |
logger.debug("{} finish_reason: {}", tag, finish_reason)
|
| 369 |
|
| 370 |
reasoning = self._provider._profile.reasoning_delta(delta)
|
| 371 |
+
if output_reasoning and reasoning is not None:
|
| 372 |
for event in hold_events(ledger.ensure_thinking_block()):
|
| 373 |
yield event
|
| 374 |
if reasoning:
|
|
|
|
| 380 |
for event in self._provider._handle_extra_reasoning(
|
| 381 |
delta,
|
| 382 |
ledger,
|
| 383 |
+
output_reasoning=output_reasoning,
|
| 384 |
):
|
| 385 |
for out_event in hold_event(event):
|
| 386 |
yield out_event
|
|
|
|
| 388 |
if delta.content:
|
| 389 |
for part in think_parser.feed(delta.content):
|
| 390 |
if part.type == ContentType.THINKING:
|
| 391 |
+
if not output_reasoning:
|
| 392 |
continue
|
| 393 |
for event in hold_events(
|
| 394 |
ledger.ensure_thinking_block()
|
|
|
|
| 483 |
ledger=ledger,
|
| 484 |
error=error,
|
| 485 |
tool_argument_alias_buffers=tool_argument_alias_buffers,
|
| 486 |
+
output_reasoning=output_reasoning,
|
| 487 |
)
|
| 488 |
except Exception as recovery_error:
|
| 489 |
trace_event(
|
|
|
|
| 556 |
remaining = think_parser.flush()
|
| 557 |
if remaining:
|
| 558 |
if remaining.type == ContentType.THINKING:
|
| 559 |
+
if not output_reasoning:
|
| 560 |
remaining = None
|
| 561 |
else:
|
| 562 |
for event in hold_events(ledger.ensure_thinking_block()):
|
|
|
|
| 707 |
ledger: AnthropicStreamLedger,
|
| 708 |
error: Exception,
|
| 709 |
tool_argument_alias_buffers: dict[int, str],
|
| 710 |
+
output_reasoning: bool,
|
| 711 |
) -> list[str] | None:
|
| 712 |
"""Build terminal recovery events when the interrupted stream permits it."""
|
| 713 |
if not is_retryable_stream_error(error):
|
|
|
|
| 749 |
|
| 750 |
recovery_body = make_text_recovery_body(body, partial_text, partial_thinking)
|
| 751 |
text, thinking = await self._collect_recovery_text(
|
| 752 |
+
recovery_body, include_reasoning=output_reasoning
|
| 753 |
)
|
| 754 |
text_suffix = continuation_suffix(partial_text, text)
|
| 755 |
thinking_suffix = continuation_suffix(partial_thinking, thinking)
|
|
@@ -0,0 +1,146 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Provider-owned reasoning translations for OpenAI-compatible APIs."""
|
| 2 |
+
|
| 3 |
+
from dataclasses import dataclass
|
| 4 |
+
from typing import Any, Protocol
|
| 5 |
+
|
| 6 |
+
from free_claude_code.core.reasoning import (
|
| 7 |
+
ReasoningControl,
|
| 8 |
+
ReasoningEffort,
|
| 9 |
+
ReasoningPolicy,
|
| 10 |
+
)
|
| 11 |
+
|
| 12 |
+
EffortValues = tuple[tuple[ReasoningEffort, str], ...]
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
class ReasoningEncoder(Protocol):
|
| 16 |
+
"""Translate provider-neutral reasoning intent into one wire shape."""
|
| 17 |
+
|
| 18 |
+
def encode(self, body: dict[str, Any], policy: ReasoningPolicy) -> None: ...
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
@dataclass(frozen=True, slots=True)
|
| 22 |
+
class NoReasoning:
|
| 23 |
+
"""Leave reasoning computation entirely to the upstream provider."""
|
| 24 |
+
|
| 25 |
+
def encode(self, body: dict[str, Any], policy: ReasoningPolicy) -> None:
|
| 26 |
+
return
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
@dataclass(frozen=True, slots=True)
|
| 30 |
+
class NamedEffortReasoning:
|
| 31 |
+
"""Encode a provider's documented named-effort vocabulary."""
|
| 32 |
+
|
| 33 |
+
efforts: EffortValues
|
| 34 |
+
disabled_value: str | bool | None = None
|
| 35 |
+
enabled_value: str | bool | None = None
|
| 36 |
+
field: str = "reasoning_effort"
|
| 37 |
+
budget_field: str | None = None
|
| 38 |
+
use_extra_body: bool = False
|
| 39 |
+
|
| 40 |
+
def encode(self, body: dict[str, Any], policy: ReasoningPolicy) -> None:
|
| 41 |
+
target = _extra_body(body) if self.use_extra_body else body
|
| 42 |
+
if policy.control is ReasoningControl.OFF:
|
| 43 |
+
if self.disabled_value is not None:
|
| 44 |
+
target[self.field] = self.disabled_value
|
| 45 |
+
return
|
| 46 |
+
|
| 47 |
+
if policy.budget_tokens is not None and self.budget_field is not None:
|
| 48 |
+
target[self.budget_field] = policy.budget_tokens
|
| 49 |
+
return
|
| 50 |
+
|
| 51 |
+
effort = dict(self.efforts).get(policy.effort)
|
| 52 |
+
if effort is not None:
|
| 53 |
+
target[self.field] = effort
|
| 54 |
+
return
|
| 55 |
+
|
| 56 |
+
if policy.control is ReasoningControl.ON and self.enabled_value is not None:
|
| 57 |
+
target[self.field] = self.enabled_value
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
@dataclass(frozen=True, slots=True)
|
| 61 |
+
class ReasoningObject:
|
| 62 |
+
"""Encode gateways that accept a top-level ``reasoning`` object."""
|
| 63 |
+
|
| 64 |
+
efforts: EffortValues
|
| 65 |
+
supports_budget: bool = True
|
| 66 |
+
|
| 67 |
+
def encode(self, body: dict[str, Any], policy: ReasoningPolicy) -> None:
|
| 68 |
+
if policy.control is ReasoningControl.OFF:
|
| 69 |
+
_extra_body(body)["reasoning"] = {"enabled": False}
|
| 70 |
+
return
|
| 71 |
+
|
| 72 |
+
reasoning: dict[str, Any] = {}
|
| 73 |
+
if policy.budget_tokens is not None and self.supports_budget:
|
| 74 |
+
reasoning["max_tokens"] = policy.budget_tokens
|
| 75 |
+
elif effort := dict(self.efforts).get(policy.effort):
|
| 76 |
+
reasoning["effort"] = effort
|
| 77 |
+
elif policy.control is ReasoningControl.ON:
|
| 78 |
+
reasoning["enabled"] = True
|
| 79 |
+
|
| 80 |
+
if reasoning:
|
| 81 |
+
_extra_body(body)["reasoning"] = reasoning
|
| 82 |
+
|
| 83 |
+
|
| 84 |
+
@dataclass(frozen=True, slots=True)
|
| 85 |
+
class ThinkingObjectReasoning:
|
| 86 |
+
"""Encode providers with an enabled/disabled ``thinking`` object."""
|
| 87 |
+
|
| 88 |
+
enabled: dict[str, Any]
|
| 89 |
+
disabled: dict[str, Any]
|
| 90 |
+
|
| 91 |
+
def encode(self, body: dict[str, Any], policy: ReasoningPolicy) -> None:
|
| 92 |
+
if policy.control is ReasoningControl.OFF:
|
| 93 |
+
_extra_body(body)["thinking"] = dict(self.disabled)
|
| 94 |
+
elif policy.requests_reasoning:
|
| 95 |
+
_extra_body(body)["thinking"] = dict(self.enabled)
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
@dataclass(frozen=True, slots=True)
|
| 99 |
+
class ChatTemplateReasoning:
|
| 100 |
+
"""Encode a provider-wide chat-template boolean without model guessing."""
|
| 101 |
+
|
| 102 |
+
field: str = "thinking"
|
| 103 |
+
|
| 104 |
+
def encode(self, body: dict[str, Any], policy: ReasoningPolicy) -> None:
|
| 105 |
+
if not policy.requests_reasoning and policy.control is not ReasoningControl.OFF:
|
| 106 |
+
return
|
| 107 |
+
kwargs = _nested_dict(_extra_body(body), "chat_template_kwargs")
|
| 108 |
+
kwargs[self.field] = policy.control is not ReasoningControl.OFF
|
| 109 |
+
|
| 110 |
+
|
| 111 |
+
@dataclass(frozen=True, slots=True)
|
| 112 |
+
class LlamaCppReasoning:
|
| 113 |
+
"""Encode llama.cpp's exact per-request numeric thinking budget."""
|
| 114 |
+
|
| 115 |
+
def encode(self, body: dict[str, Any], policy: ReasoningPolicy) -> None:
|
| 116 |
+
if policy.control is ReasoningControl.OFF:
|
| 117 |
+
_extra_body(body)["thinking_budget_tokens"] = 0
|
| 118 |
+
elif policy.budget_tokens is not None:
|
| 119 |
+
_extra_body(body)["thinking_budget_tokens"] = policy.budget_tokens
|
| 120 |
+
|
| 121 |
+
|
| 122 |
+
@dataclass(frozen=True, slots=True)
|
| 123 |
+
class SplitReasoningOutput:
|
| 124 |
+
"""Request separate reasoning output where compute is not controllable."""
|
| 125 |
+
|
| 126 |
+
def encode(self, body: dict[str, Any], policy: ReasoningPolicy) -> None:
|
| 127 |
+
_extra_body(body)["reasoning_split"] = True
|
| 128 |
+
|
| 129 |
+
|
| 130 |
+
def _extra_body(body: dict[str, Any]) -> dict[str, Any]:
|
| 131 |
+
value = body.setdefault("extra_body", {})
|
| 132 |
+
if not isinstance(value, dict):
|
| 133 |
+
raise TypeError("OpenAI extra_body must be an object.")
|
| 134 |
+
return value
|
| 135 |
+
|
| 136 |
+
|
| 137 |
+
def _nested_dict(container: dict[str, Any], key: str) -> dict[str, Any]:
|
| 138 |
+
value = container.setdefault(key, {})
|
| 139 |
+
if not isinstance(value, dict):
|
| 140 |
+
raise TypeError(f"{key} must be an object.")
|
| 141 |
+
return value
|
| 142 |
+
|
| 143 |
+
|
| 144 |
+
NO_REASONING = NoReasoning()
|
| 145 |
+
LLAMACPP_REASONING = LlamaCppReasoning()
|
| 146 |
+
SPLIT_REASONING_OUTPUT = SplitReasoningOutput()
|
|
@@ -11,9 +11,12 @@ from free_claude_code.application.errors import InvalidRequestError
|
|
| 11 |
from free_claude_code.core.anthropic import ReasoningReplayMode, build_base_request_body
|
| 12 |
from free_claude_code.core.anthropic.conversion import OpenAIConversionError
|
| 13 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
|
|
|
| 14 |
|
| 15 |
MaxTokensField = Literal["max_tokens", "max_completion_tokens"]
|
| 16 |
-
OpenAIChatPostprocessor = Callable[
|
|
|
|
|
|
|
| 17 |
ExtraBodyValidator = Callable[[dict[str, Any]], None]
|
| 18 |
|
| 19 |
|
|
@@ -22,12 +25,12 @@ class OpenAIChatRequestPolicy:
|
|
| 22 |
"""Provider policy for Anthropic-to-OpenAI chat request conversion."""
|
| 23 |
|
| 24 |
provider_name: str
|
|
|
|
| 25 |
include_extra_body: bool = False
|
| 26 |
extra_body_validator: ExtraBodyValidator | None = None
|
| 27 |
reject_extra_body_message: str | None = None
|
| 28 |
default_max_tokens: int | None = None
|
| 29 |
max_tokens_field: MaxTokensField = "max_tokens"
|
| 30 |
-
reasoning_replay: ReasoningReplayMode | None = None
|
| 31 |
strip_message_names: bool = False
|
| 32 |
unsupported_body_keys: frozenset[str] = field(default_factory=frozenset)
|
| 33 |
normalize_n_to_one: bool = False
|
|
@@ -36,8 +39,7 @@ class OpenAIChatRequestPolicy:
|
|
| 36 |
def build_openai_chat_request_body(
|
| 37 |
request_data: MessagesRequest,
|
| 38 |
*,
|
| 39 |
-
|
| 40 |
-
reasoning_history_enabled: bool | None = None,
|
| 41 |
policy: OpenAIChatRequestPolicy,
|
| 42 |
postprocessors: Iterable[OpenAIChatPostprocessor] = (),
|
| 43 |
) -> dict[str, Any]:
|
|
@@ -49,18 +51,10 @@ def build_openai_chat_request_body(
|
|
| 49 |
len(request_data.messages),
|
| 50 |
)
|
| 51 |
try:
|
| 52 |
-
if reasoning_history_enabled is None:
|
| 53 |
-
reasoning_history_enabled = thinking_enabled
|
| 54 |
-
if not reasoning_history_enabled:
|
| 55 |
-
reasoning_replay = ReasoningReplayMode.DISABLED
|
| 56 |
-
else:
|
| 57 |
-
reasoning_replay = (
|
| 58 |
-
policy.reasoning_replay or ReasoningReplayMode.REASONING_CONTENT
|
| 59 |
-
)
|
| 60 |
body = build_base_request_body(
|
| 61 |
request_data,
|
| 62 |
default_max_tokens=policy.default_max_tokens,
|
| 63 |
-
reasoning_replay=reasoning_replay,
|
| 64 |
)
|
| 65 |
except OpenAIConversionError as exc:
|
| 66 |
raise InvalidRequestError(str(exc)) from exc
|
|
@@ -81,7 +75,7 @@ def build_openai_chat_request_body(
|
|
| 81 |
_apply_common_openai_chat_policy(body, policy)
|
| 82 |
|
| 83 |
for postprocess in postprocessors:
|
| 84 |
-
postprocess(body, request_data,
|
| 85 |
|
| 86 |
logger.debug(
|
| 87 |
"{}_REQUEST: conversion done model={} msgs={} tools={}",
|
|
|
|
| 11 |
from free_claude_code.core.anthropic import ReasoningReplayMode, build_base_request_body
|
| 12 |
from free_claude_code.core.anthropic.conversion import OpenAIConversionError
|
| 13 |
from free_claude_code.core.anthropic.models import MessagesRequest
|
| 14 |
+
from free_claude_code.core.reasoning import ReasoningPolicy
|
| 15 |
|
| 16 |
MaxTokensField = Literal["max_tokens", "max_completion_tokens"]
|
| 17 |
+
OpenAIChatPostprocessor = Callable[
|
| 18 |
+
[dict[str, Any], MessagesRequest, ReasoningPolicy], None
|
| 19 |
+
]
|
| 20 |
ExtraBodyValidator = Callable[[dict[str, Any]], None]
|
| 21 |
|
| 22 |
|
|
|
|
| 25 |
"""Provider policy for Anthropic-to-OpenAI chat request conversion."""
|
| 26 |
|
| 27 |
provider_name: str
|
| 28 |
+
reasoning_replay: ReasoningReplayMode
|
| 29 |
include_extra_body: bool = False
|
| 30 |
extra_body_validator: ExtraBodyValidator | None = None
|
| 31 |
reject_extra_body_message: str | None = None
|
| 32 |
default_max_tokens: int | None = None
|
| 33 |
max_tokens_field: MaxTokensField = "max_tokens"
|
|
|
|
| 34 |
strip_message_names: bool = False
|
| 35 |
unsupported_body_keys: frozenset[str] = field(default_factory=frozenset)
|
| 36 |
normalize_n_to_one: bool = False
|
|
|
|
| 39 |
def build_openai_chat_request_body(
|
| 40 |
request_data: MessagesRequest,
|
| 41 |
*,
|
| 42 |
+
reasoning: ReasoningPolicy,
|
|
|
|
| 43 |
policy: OpenAIChatRequestPolicy,
|
| 44 |
postprocessors: Iterable[OpenAIChatPostprocessor] = (),
|
| 45 |
) -> dict[str, Any]:
|
|
|
|
| 51 |
len(request_data.messages),
|
| 52 |
)
|
| 53 |
try:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
body = build_base_request_body(
|
| 55 |
request_data,
|
| 56 |
default_max_tokens=policy.default_max_tokens,
|
| 57 |
+
reasoning_replay=policy.reasoning_replay,
|
| 58 |
)
|
| 59 |
except OpenAIConversionError as exc:
|
| 60 |
raise InvalidRequestError(str(exc)) from exc
|
|
|
|
| 75 |
_apply_common_openai_chat_policy(body, policy)
|
| 76 |
|
| 77 |
for postprocess in postprocessors:
|
| 78 |
+
postprocess(body, request_data, reasoning)
|
| 79 |
|
| 80 |
logger.debug(
|
| 81 |
"{}_REQUEST: conversion done model={} msgs={} tools={}",
|