Spaces:
Running on Zero
Running on Zero
Upload 4 files
Browse files- README_final.md +108 -0
- app_final.py +567 -0
- openai_compat_final.py +1075 -0
- requirements_final.txt +26 -0
README_final.md
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: Qwen2.5 Coder 32B AWQ OpenAI API
|
| 3 |
+
emoji: 🚀
|
| 4 |
+
colorFrom: green
|
| 5 |
+
colorTo: yellow
|
| 6 |
+
sdk: gradio
|
| 7 |
+
sdk_version: 6.22.0
|
| 8 |
+
python_version: '3.12'
|
| 9 |
+
app_file: app.py
|
| 10 |
+
pinned: false
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# Qwen2.5-Coder-32B AWQ — OpenAI-compatible ZeroGPU API
|
| 14 |
+
|
| 15 |
+
This Space serves `Qwen/Qwen2.5-Coder-32B-Instruct-AWQ` through an
|
| 16 |
+
OpenAI-compatible Chat Completions endpoint on Hugging Face ZeroGPU.
|
| 17 |
+
|
| 18 |
+
The runtime is pinned to the versions that were validated during the Space
|
| 19 |
+
startup work:
|
| 20 |
+
|
| 21 |
+
- PyTorch 2.11.0 / CUDA 13.0 wheels
|
| 22 |
+
- torchvision 0.26.0
|
| 23 |
+
- Transformers 5.14.1
|
| 24 |
+
- GPTQModel 7.3.2
|
| 25 |
+
- Gradio 6.22.0
|
| 26 |
+
|
| 27 |
+
The AWQ model is intentionally loaded lazily from inside the `@spaces.GPU`
|
| 28 |
+
function. This is required by this deployment because AWQ/Marlin performs real
|
| 29 |
+
CUDA work while `from_pretrained()` is running. Do not move model loading back
|
| 30 |
+
to module startup without retesting the Space on ZeroGPU.
|
| 31 |
+
|
| 32 |
+
## API
|
| 33 |
+
|
| 34 |
+
- `GET /health`
|
| 35 |
+
- `GET /v1/models`
|
| 36 |
+
- `POST /v1/chat/completions`
|
| 37 |
+
- `GET /web-search?q=...`
|
| 38 |
+
|
| 39 |
+
Accepted model names are the real model ID plus the compatibility aliases
|
| 40 |
+
`qwen2.5-coder-32b` and `qwen-coder`. Aliases for unrelated Qwen3 or 14B
|
| 41 |
+
weights are deliberately not accepted.
|
| 42 |
+
|
| 43 |
+
Example client configuration:
|
| 44 |
+
|
| 45 |
+
```text
|
| 46 |
+
OPENAI_BASE_URL=https://erinaldorodrigues-qwen-coder-api.hf.space/v1
|
| 47 |
+
OPENAI_API_BASE=https://erinaldorodrigues-qwen-coder-api.hf.space/v1
|
| 48 |
+
OPENAI_MODEL=qwen2.5-coder-32b
|
| 49 |
+
WEB_SEARCH_PROVIDER=custom
|
| 50 |
+
WEB_SEARCH_API=https://erinaldorodrigues-qwen-coder-api.hf.space/web-search
|
| 51 |
+
WEB_METHOD=GET
|
| 52 |
+
WEB_QUERY_PARAM=q
|
| 53 |
+
```
|
| 54 |
+
|
| 55 |
+
If a client requires a non-empty `OPENAI_API_KEY`, it may send one, but the
|
| 56 |
+
current `app.py` does not implement application-level Bearer-token validation.
|
| 57 |
+
Add authentication before exposing private quota or sensitive tools.
|
| 58 |
+
|
| 59 |
+
## Tool calling
|
| 60 |
+
|
| 61 |
+
The backend accepts OpenAI-style `tools`, `tool_choice`, and
|
| 62 |
+
`parallel_tool_calls`. Tool definitions are normalized for Qwen's native chat
|
| 63 |
+
template and textual `<tool_call>...</tool_call>` outputs are translated back
|
| 64 |
+
to OpenAI `message.tool_calls` objects.
|
| 65 |
+
|
| 66 |
+
`tool_choice="required"` is reconciled with conversation state for OpenClaude:
|
| 67 |
+
when the user explicitly identifies a tool, the choice is narrowed to that
|
| 68 |
+
function; once usable tool evidence is already present, repeated `required`
|
| 69 |
+
choices may be downgraded to `none` so the model can synthesize the answer
|
| 70 |
+
instead of looping.
|
| 71 |
+
|
| 72 |
+
When `parallel_tool_calls=true`, the prompt permits multiple independent tool
|
| 73 |
+
calls. Otherwise generation stops after the first complete tool call.
|
| 74 |
+
|
| 75 |
+
The Space generates tool calls; the calling client remains responsible for
|
| 76 |
+
executing client-side tools and returning their results in subsequent `tool`
|
| 77 |
+
messages. `/web-search` is a separate server-side search endpoint.
|
| 78 |
+
|
| 79 |
+
## Generation and compatibility
|
| 80 |
+
|
| 81 |
+
The default context limit is 16,384 tokens and default maximum output is 2,048
|
| 82 |
+
tokens. Both can be changed with Space environment variables.
|
| 83 |
+
|
| 84 |
+
`temperature=0` is preserved as greedy generation. Responses report prompt and
|
| 85 |
+
completion token counts, and `finish_reason="length"` is returned when the
|
| 86 |
+
configured output budget is exhausted without a completed tool call.
|
| 87 |
+
|
| 88 |
+
`stream=true` returns OpenAI-style SSE framing. The current implementation
|
| 89 |
+
finishes model generation before emitting the content delta, so it is protocol
|
| 90 |
+
streaming rather than token-by-token low-latency streaming. This is intentional
|
| 91 |
+
until a ZeroGPU-safe streamer/thread implementation is validated.
|
| 92 |
+
|
| 93 |
+
## Health
|
| 94 |
+
|
| 95 |
+
`/health` reports both the configured model and `model_loaded`. Because model
|
| 96 |
+
loading is lazy, a healthy freshly started process can report
|
| 97 |
+
`model_loaded=false` until the first GPU inference initializes the AWQ model.
|
| 98 |
+
|
| 99 |
+
## Tests
|
| 100 |
+
|
| 101 |
+
Run:
|
| 102 |
+
|
| 103 |
+
```bash
|
| 104 |
+
python -m unittest discover -p 'test_*.py'
|
| 105 |
+
```
|
| 106 |
+
|
| 107 |
+
The suite covers generation helpers, OpenAI/OpenClaude tool flow, tool-call
|
| 108 |
+
parsing, web-search fallbacks, and static integration regressions in `app.py`.
|
app_final.py
ADDED
|
@@ -0,0 +1,567 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Reliable ZeroGPU backend for the local OpenAI-compatible proxy."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import json
|
| 6 |
+
import os
|
| 7 |
+
import time
|
| 8 |
+
import uuid
|
| 9 |
+
import traceback
|
| 10 |
+
from typing import Any
|
| 11 |
+
|
| 12 |
+
os.environ.setdefault("HF_HUB_DISABLE_PROGRESS_BARS", "1")
|
| 13 |
+
os.environ.setdefault("TRANSFORMERS_DISABLE_DEEPGEMM_LINEAR", "1")
|
| 14 |
+
|
| 15 |
+
import gradio as gr
|
| 16 |
+
import spaces
|
| 17 |
+
import torch
|
| 18 |
+
from fastapi import HTTPException
|
| 19 |
+
from fastapi.responses import JSONResponse, StreamingResponse
|
| 20 |
+
from pydantic import BaseModel, ValidationError
|
| 21 |
+
from starlette.concurrency import run_in_threadpool
|
| 22 |
+
from starlette.middleware.base import BaseHTTPMiddleware
|
| 23 |
+
from starlette.requests import Request
|
| 24 |
+
from transformers import (
|
| 25 |
+
AutoModelForCausalLM,
|
| 26 |
+
AutoTokenizer,
|
| 27 |
+
StoppingCriteria,
|
| 28 |
+
StoppingCriteriaList,
|
| 29 |
+
)
|
| 30 |
+
from generation import (
|
| 31 |
+
gpu_duration_seconds,
|
| 32 |
+
head_tail_token_counts,
|
| 33 |
+
merge_eos_token_ids,
|
| 34 |
+
)
|
| 35 |
+
from openai_compat import (
|
| 36 |
+
analyze_tool_flow,
|
| 37 |
+
indexed_tool_calls,
|
| 38 |
+
normalize_tools,
|
| 39 |
+
resolve_tool_choice,
|
| 40 |
+
select_tools,
|
| 41 |
+
tool_choice_instruction,
|
| 42 |
+
tool_protocol_instruction,
|
| 43 |
+
tool_names,
|
| 44 |
+
)
|
| 45 |
+
from openclaude_compat import (
|
| 46 |
+
TOOL_PROTOCOL_MARKER,
|
| 47 |
+
add_system_instruction,
|
| 48 |
+
has_tool_protocol,
|
| 49 |
+
normalize_openclaude_messages,
|
| 50 |
+
)
|
| 51 |
+
from tool_calls import (
|
| 52 |
+
extract_tool_calls,
|
| 53 |
+
has_complete_tool_call,
|
| 54 |
+
)
|
| 55 |
+
from web_search import SearchUnavailable, search_web
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
# O Titã: Qwen2.5-Coder-32B nativamente quantizado em 4-bits (AWQ)
|
| 59 |
+
MODEL = os.getenv(
|
| 60 |
+
"MODEL",
|
| 61 |
+
os.getenv("MODEL_ID", "Qwen/Qwen2.5-Coder-32B-Instruct-AWQ"),
|
| 62 |
+
)
|
| 63 |
+
|
| 64 |
+
MAX_CONTEXT_TOKENS = int(os.getenv("MAX_CONTEXT_TOKENS", "16384"))
|
| 65 |
+
MAX_NEW_TOKENS = int(os.getenv("MAX_NEW_TOKENS", "2048"))
|
| 66 |
+
MAX_TOOL_CALL_TOKENS = int(os.getenv("MAX_TOOL_CALL_TOKENS", "2048"))
|
| 67 |
+
MAX_TEMPERATURE = float(os.getenv("MAX_TEMPERATURE", "0.2"))
|
| 68 |
+
PRESERVED_PREFIX_TOKENS = int(os.getenv("PRESERVED_PREFIX_TOKENS", "4096"))
|
| 69 |
+
|
| 70 |
+
tokenizer = AutoTokenizer.from_pretrained(MODEL)
|
| 71 |
+
|
| 72 |
+
# O ZeroGPU só anexa uma GPU real dentro de funções decoradas com
|
| 73 |
+
# @spaces.GPU; no escopo do módulo (startup) não existe CUDA de verdade,
|
| 74 |
+
# apenas uma emulação que aceita `.to("cuda")`/`device_map="auto"` como
|
| 75 |
+
# simples posicionamento de tensores. O carregamento deste modelo AWQ,
|
| 76 |
+
# porém, dispara o kernel Marlin (`awq_marlin_repack`) de forma síncrona
|
| 77 |
+
# dentro do próprio from_pretrained — isso é execução real de kernel CUDA,
|
| 78 |
+
# não posicionamento, e por isso não existe backend CPU para ele (era
|
| 79 |
+
# exatamente esse o erro do seu log). Por isso o carregamento precisa ser
|
| 80 |
+
# adiado para dentro de `gerar`, a única função com GPU real anexada.
|
| 81 |
+
model: AutoModelForCausalLM | None = None
|
| 82 |
+
|
| 83 |
+
|
| 84 |
+
def _ensure_model_loaded() -> None:
|
| 85 |
+
"""Carrega o modelo uma única vez, já dentro do contexto com GPU real."""
|
| 86 |
+
global model
|
| 87 |
+
if model is not None:
|
| 88 |
+
return
|
| 89 |
+
print(f"Loading {MODEL} on ZeroGPU (NATIVE AWQ)...", flush=True)
|
| 90 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 91 |
+
MODEL,
|
| 92 |
+
dtype="auto",
|
| 93 |
+
device_map="auto",
|
| 94 |
+
low_cpu_mem_usage=True,
|
| 95 |
+
)
|
| 96 |
+
model.eval()
|
| 97 |
+
print(f"Model ready on {next(model.parameters()).device}", flush=True)
|
| 98 |
+
|
| 99 |
+
|
| 100 |
+
def _bounded_output_tokens(value: float) -> int:
|
| 101 |
+
try:
|
| 102 |
+
requested = int(value)
|
| 103 |
+
except (TypeError, ValueError):
|
| 104 |
+
requested = MAX_NEW_TOKENS
|
| 105 |
+
return max(1, min(requested, MAX_NEW_TOKENS))
|
| 106 |
+
|
| 107 |
+
|
| 108 |
+
# Buffer para cobrir a compilação JIT do kernel Marlin + carregamento dos
|
| 109 |
+
# pesos quando `gerar` cai num worker "frio" (sem o modelo em memória).
|
| 110 |
+
# É uma estimativa (baseada nos ~99s de compilação que aparecem no seu log);
|
| 111 |
+
# meça o cold start real do seu Space e ajuste. Confira também o teto de
|
| 112 |
+
# duração por chamada da sua tier em
|
| 113 |
+
# https://huggingface.co/docs/hub/spaces-zerogpu antes de subir esse valor —
|
| 114 |
+
# se o teto for menor que isso, a chamada falha com "illegal duration".
|
| 115 |
+
COLD_START_BUFFER_SECONDS = 180
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
def _gpu_duration(
|
| 119 |
+
messages_json: str,
|
| 120 |
+
__: float,
|
| 121 |
+
max_new_tokens: float,
|
| 122 |
+
*tool_arguments: object,
|
| 123 |
+
) -> int:
|
| 124 |
+
output_tokens = _bounded_output_tokens(max_new_tokens)
|
| 125 |
+
tool_characters = sum(
|
| 126 |
+
len(value) for value in tool_arguments if isinstance(value, str)
|
| 127 |
+
)
|
| 128 |
+
duration = gpu_duration_seconds(
|
| 129 |
+
len(messages_json) + tool_characters,
|
| 130 |
+
output_tokens,
|
| 131 |
+
MAX_CONTEXT_TOKENS,
|
| 132 |
+
)
|
| 133 |
+
return duration + COLD_START_BUFFER_SECONDS
|
| 134 |
+
|
| 135 |
+
|
| 136 |
+
def _tool_protocol_active(messages: list[object]) -> bool:
|
| 137 |
+
return any(
|
| 138 |
+
isinstance(message, dict)
|
| 139 |
+
and isinstance(message.get("content"), str)
|
| 140 |
+
and TOOL_PROTOCOL_MARKER in message["content"]
|
| 141 |
+
for message in messages
|
| 142 |
+
)
|
| 143 |
+
|
| 144 |
+
|
| 145 |
+
def _native_tools(raw_tools: object) -> list[dict[str, Any]]:
|
| 146 |
+
return normalize_tools(raw_tools)
|
| 147 |
+
|
| 148 |
+
|
| 149 |
+
def _render_prompt(
|
| 150 |
+
messages: list[dict[str, Any]],
|
| 151 |
+
tools: list[dict[str, Any]],
|
| 152 |
+
) -> str:
|
| 153 |
+
"""Render the exact prompt used for generation, with a safe tool fallback."""
|
| 154 |
+
template_kwargs: dict[str, Any] = {
|
| 155 |
+
"tokenize": False,
|
| 156 |
+
"add_generation_prompt": True,
|
| 157 |
+
}
|
| 158 |
+
if tools:
|
| 159 |
+
template_kwargs["tools"] = tools
|
| 160 |
+
|
| 161 |
+
try:
|
| 162 |
+
return tokenizer.apply_chat_template(messages, **template_kwargs)
|
| 163 |
+
except Exception as template_error:
|
| 164 |
+
print(
|
| 165 |
+
f"Jinja Template Warning: {template_error}. Applying fallback.",
|
| 166 |
+
flush=True,
|
| 167 |
+
)
|
| 168 |
+
template_kwargs.pop("tools", None)
|
| 169 |
+
return tokenizer.apply_chat_template(messages, **template_kwargs)
|
| 170 |
+
|
| 171 |
+
|
| 172 |
+
def _prompt_token_count(
|
| 173 |
+
messages: list[dict[str, Any]],
|
| 174 |
+
tools: list[dict[str, Any]],
|
| 175 |
+
output_tokens: int,
|
| 176 |
+
) -> int:
|
| 177 |
+
"""Count the prompt tokens that survive the same context bound as generation."""
|
| 178 |
+
prompt = _render_prompt(messages, tools)
|
| 179 |
+
encoded = tokenizer(
|
| 180 |
+
prompt,
|
| 181 |
+
add_special_tokens=False,
|
| 182 |
+
truncation=False,
|
| 183 |
+
)["input_ids"]
|
| 184 |
+
input_budget = max(1, MAX_CONTEXT_TOKENS - output_tokens)
|
| 185 |
+
return min(len(encoded), input_budget)
|
| 186 |
+
|
| 187 |
+
|
| 188 |
+
def _completion_token_count(text: str) -> int:
|
| 189 |
+
"""Count visible generated tokens for OpenAI-compatible usage reporting."""
|
| 190 |
+
return len(
|
| 191 |
+
tokenizer(
|
| 192 |
+
text,
|
| 193 |
+
add_special_tokens=False,
|
| 194 |
+
truncation=False,
|
| 195 |
+
)["input_ids"]
|
| 196 |
+
)
|
| 197 |
+
|
| 198 |
+
|
| 199 |
+
class StopAfterToolCall(StoppingCriteria):
|
| 200 |
+
def __init__(self, prompt_length: int) -> None:
|
| 201 |
+
self.prompt_length = prompt_length
|
| 202 |
+
|
| 203 |
+
def __call__(self, input_ids, scores, **_: object):
|
| 204 |
+
completed = []
|
| 205 |
+
for sequence in input_ids:
|
| 206 |
+
generated = sequence[self.prompt_length :]
|
| 207 |
+
text = tokenizer.decode(generated, skip_special_tokens=False)
|
| 208 |
+
completed.append(has_complete_tool_call(text))
|
| 209 |
+
return torch.tensor(completed, dtype=torch.bool, device=input_ids.device)
|
| 210 |
+
|
| 211 |
+
|
| 212 |
+
@spaces.GPU(duration=_gpu_duration)
|
| 213 |
+
def gerar(
|
| 214 |
+
messages_json: str,
|
| 215 |
+
temperature: float,
|
| 216 |
+
max_new_tokens: float,
|
| 217 |
+
tools_json: str = "[]",
|
| 218 |
+
stop_after_first_tool: bool = True,
|
| 219 |
+
) -> str:
|
| 220 |
+
_ensure_model_loaded()
|
| 221 |
+
messages = json.loads(messages_json)
|
| 222 |
+
if not isinstance(messages, list):
|
| 223 |
+
raise ValueError("messages_json must contain a JSON list")
|
| 224 |
+
try:
|
| 225 |
+
tools = _native_tools(json.loads(tools_json))
|
| 226 |
+
except (TypeError, ValueError, json.JSONDecodeError):
|
| 227 |
+
tools = []
|
| 228 |
+
if not isinstance(tools, list):
|
| 229 |
+
tools = []
|
| 230 |
+
|
| 231 |
+
output_tokens = _bounded_output_tokens(max_new_tokens)
|
| 232 |
+
tool_mode = _tool_protocol_active(messages) or bool(tools)
|
| 233 |
+
prompt = _render_prompt(messages, tools)
|
| 234 |
+
|
| 235 |
+
inputs = tokenizer(
|
| 236 |
+
prompt,
|
| 237 |
+
return_tensors="pt",
|
| 238 |
+
add_special_tokens=False,
|
| 239 |
+
truncation=False,
|
| 240 |
+
)
|
| 241 |
+
input_budget = max(1, MAX_CONTEXT_TOKENS - output_tokens)
|
| 242 |
+
input_length = inputs["input_ids"].shape[1]
|
| 243 |
+
|
| 244 |
+
if input_length > input_budget:
|
| 245 |
+
head_tokens, tail_tokens = head_tail_token_counts(
|
| 246 |
+
input_length,
|
| 247 |
+
input_budget,
|
| 248 |
+
PRESERVED_PREFIX_TOKENS,
|
| 249 |
+
)
|
| 250 |
+
for key, value in inputs.items():
|
| 251 |
+
if (
|
| 252 |
+
isinstance(value, torch.Tensor)
|
| 253 |
+
and value.ndim == 2
|
| 254 |
+
and value.shape[1] == input_length
|
| 255 |
+
):
|
| 256 |
+
parts = []
|
| 257 |
+
if head_tokens:
|
| 258 |
+
parts.append(value[:, :head_tokens])
|
| 259 |
+
if tail_tokens:
|
| 260 |
+
parts.append(value[:, -tail_tokens:])
|
| 261 |
+
inputs[key] = torch.cat(parts, dim=1)
|
| 262 |
+
|
| 263 |
+
inputs = inputs.to("cuda")
|
| 264 |
+
|
| 265 |
+
print(
|
| 266 |
+
f"Generation started: input_tokens={inputs['input_ids'].shape[1]} "
|
| 267 |
+
f"max_new_tokens={output_tokens} tool_mode={tool_mode}",
|
| 268 |
+
flush=True,
|
| 269 |
+
)
|
| 270 |
+
|
| 271 |
+
eos_token_ids = merge_eos_token_ids(
|
| 272 |
+
model.generation_config.eos_token_id,
|
| 273 |
+
tokenizer.eos_token_id,
|
| 274 |
+
)
|
| 275 |
+
|
| 276 |
+
generation_kwargs = {
|
| 277 |
+
"max_new_tokens": output_tokens,
|
| 278 |
+
"do_sample": float(temperature) > 0,
|
| 279 |
+
"pad_token_id": tokenizer.pad_token_id or tokenizer.eos_token_id,
|
| 280 |
+
}
|
| 281 |
+
if eos_token_ids is not None:
|
| 282 |
+
generation_kwargs["eos_token_id"] = eos_token_ids
|
| 283 |
+
|
| 284 |
+
if generation_kwargs["do_sample"]:
|
| 285 |
+
generation_kwargs["temperature"] = max(0.01, float(temperature))
|
| 286 |
+
generation_kwargs["top_p"] = 0.8
|
| 287 |
+
generation_kwargs["top_k"] = 20
|
| 288 |
+
generation_kwargs["repetition_penalty"] = 1.05
|
| 289 |
+
|
| 290 |
+
if tool_mode and stop_after_first_tool:
|
| 291 |
+
generation_kwargs["stopping_criteria"] = StoppingCriteriaList(
|
| 292 |
+
[StopAfterToolCall(inputs["input_ids"].shape[1])]
|
| 293 |
+
)
|
| 294 |
+
|
| 295 |
+
with torch.inference_mode():
|
| 296 |
+
output = model.generate(**inputs, **generation_kwargs)
|
| 297 |
+
|
| 298 |
+
generated = output[0][inputs["input_ids"].shape[1] :]
|
| 299 |
+
response = tokenizer.decode(generated, skip_special_tokens=True).strip()
|
| 300 |
+
print(f"Generation completed: output_tokens={generated.shape[0]}", flush=True)
|
| 301 |
+
return response
|
| 302 |
+
|
| 303 |
+
|
| 304 |
+
class ChatCompletionRequest(BaseModel):
|
| 305 |
+
model: str = MODEL
|
| 306 |
+
messages: list[dict[str, Any]]
|
| 307 |
+
temperature: float = 0.2
|
| 308 |
+
max_tokens: int | None = None
|
| 309 |
+
max_completion_tokens: int | None = None
|
| 310 |
+
stream: bool = False
|
| 311 |
+
tools: list[dict[str, Any]] | None = None
|
| 312 |
+
tool_choice: Any = None
|
| 313 |
+
parallel_tool_calls: bool | None = None
|
| 314 |
+
|
| 315 |
+
|
| 316 |
+
def _completion_payload(request: ChatCompletionRequest) -> dict[str, Any]:
|
| 317 |
+
if request.model not in {
|
| 318 |
+
MODEL,
|
| 319 |
+
"qwen-coder",
|
| 320 |
+
"qwen2.5-coder-32b",
|
| 321 |
+
}:
|
| 322 |
+
raise HTTPException(status_code=404, detail=f"Model not available: {request.model}")
|
| 323 |
+
|
| 324 |
+
already_adapted = has_tool_protocol(request.messages)
|
| 325 |
+
flow_state = analyze_tool_flow(request.messages, request.tools or [])
|
| 326 |
+
state_controls_choice = request.tool_choice is None or (
|
| 327 |
+
isinstance(request.tool_choice, str)
|
| 328 |
+
and request.tool_choice.casefold() in {"auto", "required"}
|
| 329 |
+
)
|
| 330 |
+
effective_choice = resolve_tool_choice(request.tool_choice, flow_state)
|
| 331 |
+
|
| 332 |
+
try:
|
| 333 |
+
effective_tools, tool_mode = select_tools(
|
| 334 |
+
request.tools or [], effective_choice
|
| 335 |
+
)
|
| 336 |
+
except ValueError as error:
|
| 337 |
+
raise HTTPException(status_code=400, detail=str(error)) from error
|
| 338 |
+
|
| 339 |
+
instructions = [
|
| 340 |
+
instruction
|
| 341 |
+
for instruction in (
|
| 342 |
+
(
|
| 343 |
+
tool_protocol_instruction(
|
| 344 |
+
effective_tools,
|
| 345 |
+
parallel_tool_calls=request.parallel_tool_calls is True,
|
| 346 |
+
)
|
| 347 |
+
if effective_tools and not has_tool_protocol(request.messages)
|
| 348 |
+
else None
|
| 349 |
+
),
|
| 350 |
+
tool_choice_instruction(tool_mode, effective_tools),
|
| 351 |
+
(
|
| 352 |
+
flow_state.instruction
|
| 353 |
+
if state_controls_choice and not already_adapted
|
| 354 |
+
else None
|
| 355 |
+
),
|
| 356 |
+
)
|
| 357 |
+
if instruction
|
| 358 |
+
]
|
| 359 |
+
instruction = "\n\n".join(instructions) if instructions else None
|
| 360 |
+
max_tokens = request.max_completion_tokens or request.max_tokens or MAX_NEW_TOKENS
|
| 361 |
+
|
| 362 |
+
if effective_tools:
|
| 363 |
+
max_tokens = min(max_tokens, MAX_TOOL_CALL_TOKENS)
|
| 364 |
+
temperature = min(max(float(request.temperature), 0.0), MAX_TEMPERATURE)
|
| 365 |
+
|
| 366 |
+
try:
|
| 367 |
+
normalized_messages = (
|
| 368 |
+
[dict(message) for message in request.messages]
|
| 369 |
+
if already_adapted
|
| 370 |
+
else normalize_openclaude_messages(request.messages)
|
| 371 |
+
)
|
| 372 |
+
prompt_messages = add_system_instruction(
|
| 373 |
+
normalized_messages,
|
| 374 |
+
instruction,
|
| 375 |
+
)
|
| 376 |
+
except ValueError as error:
|
| 377 |
+
raise HTTPException(status_code=400, detail=str(error)) from error
|
| 378 |
+
|
| 379 |
+
bounded_max_tokens = _bounded_output_tokens(max_tokens)
|
| 380 |
+
prompt_tokens = _prompt_token_count(
|
| 381 |
+
prompt_messages,
|
| 382 |
+
effective_tools,
|
| 383 |
+
bounded_max_tokens,
|
| 384 |
+
)
|
| 385 |
+
text = gerar(
|
| 386 |
+
json.dumps(prompt_messages),
|
| 387 |
+
temperature,
|
| 388 |
+
bounded_max_tokens,
|
| 389 |
+
json.dumps(effective_tools, ensure_ascii=False),
|
| 390 |
+
request.parallel_tool_calls is not True,
|
| 391 |
+
)
|
| 392 |
+
completion_tokens = _completion_token_count(text)
|
| 393 |
+
|
| 394 |
+
if effective_tools:
|
| 395 |
+
tool_calls, content = extract_tool_calls(text, tool_names(effective_tools))
|
| 396 |
+
if request.parallel_tool_calls is False:
|
| 397 |
+
tool_calls = tool_calls[:1]
|
| 398 |
+
else:
|
| 399 |
+
tool_calls, content = [], text
|
| 400 |
+
|
| 401 |
+
message: dict[str, Any] = {"role": "assistant", "content": content or None}
|
| 402 |
+
|
| 403 |
+
finish_reason = "stop"
|
| 404 |
+
if tool_calls:
|
| 405 |
+
message["tool_calls"] = tool_calls
|
| 406 |
+
finish_reason = "tool_calls"
|
| 407 |
+
elif completion_tokens >= bounded_max_tokens:
|
| 408 |
+
finish_reason = "length"
|
| 409 |
+
elif effective_tools and has_complete_tool_call(text):
|
| 410 |
+
finish_reason = "stop"
|
| 411 |
+
elif tool_mode in {"required", "forced"}:
|
| 412 |
+
finish_reason = "stop"
|
| 413 |
+
|
| 414 |
+
return {
|
| 415 |
+
"id": f"chatcmpl-{uuid.uuid4().hex}",
|
| 416 |
+
"object": "chat.completion",
|
| 417 |
+
"created": int(time.time()),
|
| 418 |
+
"model": MODEL,
|
| 419 |
+
"choices": [{"index": 0, "message": message, "finish_reason": finish_reason}],
|
| 420 |
+
"usage": {
|
| 421 |
+
"prompt_tokens": prompt_tokens,
|
| 422 |
+
"completion_tokens": completion_tokens,
|
| 423 |
+
"total_tokens": prompt_tokens + completion_tokens,
|
| 424 |
+
},
|
| 425 |
+
}
|
| 426 |
+
|
| 427 |
+
|
| 428 |
+
def health() -> dict[str, Any]:
|
| 429 |
+
return {
|
| 430 |
+
"status": "ok",
|
| 431 |
+
"model": MODEL,
|
| 432 |
+
"model_loaded": model is not None,
|
| 433 |
+
}
|
| 434 |
+
|
| 435 |
+
|
| 436 |
+
def models() -> dict[str, Any]:
|
| 437 |
+
return {
|
| 438 |
+
"object": "list",
|
| 439 |
+
"data": [
|
| 440 |
+
{
|
| 441 |
+
"id": model_id,
|
| 442 |
+
"object": "model",
|
| 443 |
+
"owned_by": "Erinaldorodrigues",
|
| 444 |
+
"context_length": MAX_CONTEXT_TOKENS,
|
| 445 |
+
"max_input_tokens": MAX_CONTEXT_TOKENS,
|
| 446 |
+
"max_output_tokens": MAX_NEW_TOKENS,
|
| 447 |
+
}
|
| 448 |
+
for model_id in dict.fromkeys(("qwen2.5-coder-32b", MODEL))
|
| 449 |
+
],
|
| 450 |
+
}
|
| 451 |
+
|
| 452 |
+
|
| 453 |
+
def chat_completions(request: ChatCompletionRequest):
|
| 454 |
+
completion = _completion_payload(request)
|
| 455 |
+
if not request.stream:
|
| 456 |
+
return JSONResponse(content=completion)
|
| 457 |
+
|
| 458 |
+
choice = completion["choices"][0]
|
| 459 |
+
chunk_id = completion["id"]
|
| 460 |
+
|
| 461 |
+
def events():
|
| 462 |
+
first = {
|
| 463 |
+
"id": chunk_id,
|
| 464 |
+
"object": "chat.completion.chunk",
|
| 465 |
+
"created": completion["created"],
|
| 466 |
+
"model": MODEL,
|
| 467 |
+
"choices": [{"index": 0, "delta": {"role": "assistant"}, "finish_reason": None}],
|
| 468 |
+
}
|
| 469 |
+
yield f"data: {json.dumps(first)}\n\n"
|
| 470 |
+
|
| 471 |
+
delta: dict[str, Any] = {}
|
| 472 |
+
if choice["message"].get("content"):
|
| 473 |
+
delta["content"] = choice["message"]["content"]
|
| 474 |
+
if choice["message"].get("tool_calls"):
|
| 475 |
+
delta["tool_calls"] = indexed_tool_calls(
|
| 476 |
+
choice["message"]["tool_calls"]
|
| 477 |
+
)
|
| 478 |
+
|
| 479 |
+
body = {**first, "choices": [{"index": 0, "delta": delta, "finish_reason": None}]}
|
| 480 |
+
yield f"data: {json.dumps(body)}\n\n"
|
| 481 |
+
|
| 482 |
+
final = {**first, "choices": [{"index": 0, "delta": {}, "finish_reason": choice["finish_reason"]}]}
|
| 483 |
+
yield f"data: {json.dumps(final)}\n\n"
|
| 484 |
+
yield "data: [DONE]\n\n"
|
| 485 |
+
|
| 486 |
+
return StreamingResponse(
|
| 487 |
+
events(),
|
| 488 |
+
media_type="text/event-stream",
|
| 489 |
+
headers={
|
| 490 |
+
"Cache-Control": "no-cache",
|
| 491 |
+
"X-Accel-Buffering": "no",
|
| 492 |
+
},
|
| 493 |
+
)
|
| 494 |
+
|
| 495 |
+
|
| 496 |
+
demo = gr.Interface(
|
| 497 |
+
fn=gerar,
|
| 498 |
+
inputs=[
|
| 499 |
+
gr.Textbox(label="Messages JSON"),
|
| 500 |
+
gr.Number(value=0.2, label="Temperature"),
|
| 501 |
+
gr.Number(value=512, label="Max Tokens"),
|
| 502 |
+
gr.Textbox(value="[]", label="Tools JSON"),
|
| 503 |
+
gr.Checkbox(value=True, label="Stop after first complete tool call"),
|
| 504 |
+
],
|
| 505 |
+
outputs="text",
|
| 506 |
+
title="Qwen2.5-Coder-32B AWQ OpenAI-compatible ZeroGPU Backend",
|
| 507 |
+
)
|
| 508 |
+
|
| 509 |
+
class OpenAIRouteMiddleware(BaseHTTPMiddleware):
|
| 510 |
+
async def dispatch(self, request: Request, call_next):
|
| 511 |
+
path = request.url.path.rstrip("/") or "/"
|
| 512 |
+
|
| 513 |
+
if path == "/health" and request.method == "GET":
|
| 514 |
+
return JSONResponse(health())
|
| 515 |
+
|
| 516 |
+
if path == "/web-search" and request.method == "GET":
|
| 517 |
+
query = request.query_params.get("q", "").strip()
|
| 518 |
+
if not query or len(query) > 500:
|
| 519 |
+
return JSONResponse(status_code=400, content={"error": "invalid query"})
|
| 520 |
+
try:
|
| 521 |
+
return JSONResponse(await run_in_threadpool(search_web, query))
|
| 522 |
+
except SearchUnavailable as error:
|
| 523 |
+
return JSONResponse(
|
| 524 |
+
status_code=503,
|
| 525 |
+
content={"error": {"message": str(error) or "search unavailable"}},
|
| 526 |
+
)
|
| 527 |
+
except Exception as error:
|
| 528 |
+
traceback.print_exc()
|
| 529 |
+
return JSONResponse(
|
| 530 |
+
status_code=500,
|
| 531 |
+
content={"error": {"message": f"search error: {error}"}},
|
| 532 |
+
)
|
| 533 |
+
|
| 534 |
+
if path == "/v1/models" and request.method == "GET":
|
| 535 |
+
return JSONResponse(models())
|
| 536 |
+
|
| 537 |
+
if path == "/v1/chat/completions" and request.method == "POST":
|
| 538 |
+
try:
|
| 539 |
+
raw_request = await request.json()
|
| 540 |
+
parsed_request = ChatCompletionRequest(**raw_request)
|
| 541 |
+
except (json.JSONDecodeError, ValidationError, TypeError) as error:
|
| 542 |
+
return JSONResponse(status_code=400, content={"error": {"message": str(error)}})
|
| 543 |
+
try:
|
| 544 |
+
return chat_completions(parsed_request)
|
| 545 |
+
except HTTPException as error:
|
| 546 |
+
return JSONResponse(status_code=error.status_code, content={"error": {"message": error.detail}})
|
| 547 |
+
except Exception as error:
|
| 548 |
+
traceback.print_exc()
|
| 549 |
+
return JSONResponse(
|
| 550 |
+
status_code=500,
|
| 551 |
+
content={"error": {"message": f"internal Space error: {str(error)}"}}
|
| 552 |
+
)
|
| 553 |
+
|
| 554 |
+
return await call_next(request)
|
| 555 |
+
|
| 556 |
+
|
| 557 |
+
import gradio.routes as _groutes
|
| 558 |
+
_original_create_app = _groutes.App.create_app
|
| 559 |
+
|
| 560 |
+
def _create_app_with_openai_routes(*args, **kwargs):
|
| 561 |
+
created = _original_create_app(*args, **kwargs)
|
| 562 |
+
created.add_middleware(OpenAIRouteMiddleware)
|
| 563 |
+
return created
|
| 564 |
+
|
| 565 |
+
_groutes.App.create_app = staticmethod(_create_app_with_openai_routes)
|
| 566 |
+
|
| 567 |
+
demo.queue(default_concurrency_limit=1, max_size=8).launch(show_error=True, ssr_mode=False)
|
openai_compat_final.py
ADDED
|
@@ -0,0 +1,1075 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Pure OpenAI compatibility helpers used by the Space endpoint."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import json
|
| 6 |
+
import re
|
| 7 |
+
from collections.abc import Mapping
|
| 8 |
+
from dataclasses import dataclass
|
| 9 |
+
from typing import Any
|
| 10 |
+
|
| 11 |
+
from tool_calls import normalize_openai_tool_arguments
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
EMPTY_PARAMETERS = {"type": "object", "properties": {}}
|
| 15 |
+
# OpenClaude includes human-facing operational manuals in tool descriptions.
|
| 16 |
+
# They are useful to its native client but can consume most of the Qwen context
|
| 17 |
+
# once the same catalog is rendered again in the model prompt. Keep enough
|
| 18 |
+
# context to select and call a tool while preserving the full JSON-schema shape.
|
| 19 |
+
MAX_TOOL_DESCRIPTION_CHARS = 800
|
| 20 |
+
MAX_SCHEMA_DESCRIPTION_CHARS = 240
|
| 21 |
+
|
| 22 |
+
FAILED_RESULT_RE = re.compile(
|
| 23 |
+
r"(?im)(?:"
|
| 24 |
+
r"<tool_use_error>|"
|
| 25 |
+
r"\bexit\s*(?:code)?\s*[:=]?\s*[1-9]\d*\b|"
|
| 26 |
+
r"\bstatus\s*(?:code)?\s*[:=]?\s*[345]\d\d\b|"
|
| 27 |
+
r"^\s*(?:FAILED|ERROR)(?:\s|:)|"
|
| 28 |
+
r"\b[1-9]\d*\s+(?:failed|errors?)\b|"
|
| 29 |
+
r"\b(?:command not found|no such file|permission denied|timed out)\b|"
|
| 30 |
+
r"\b(?:invalid api key|invalid token|unauthorized|forbidden)\b|"
|
| 31 |
+
r"\b(?:invalid tool parameters|inputvalidationerror)\b|"
|
| 32 |
+
r"\b(?:required parameter|schema)[^\n]*(?:missing|not sent)\b|"
|
| 33 |
+
r'"status"\s*:\s*"(?:error|401|403)"|'
|
| 34 |
+
r'"status"\s*:\s*(?:401|403)\b|'
|
| 35 |
+
r"\bHTTP/\S+\s+(?:3\d\d|4\d\d|5\d\d)\b"
|
| 36 |
+
r")"
|
| 37 |
+
)
|
| 38 |
+
VERIFICATION_COMMAND_RE = re.compile(
|
| 39 |
+
r"(?i)(?:"
|
| 40 |
+
r"\bpytest\b|"
|
| 41 |
+
r"\bpython(?:3)?\s+-m\s+(?:unittest|pytest)\b|"
|
| 42 |
+
r"\bpython(?:3)?\s+[^\n;&|]*test[^\n;&|]*\.py\b|"
|
| 43 |
+
r"\b(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?test\b|"
|
| 44 |
+
r"\b(?:cargo|go)\s+test\b|"
|
| 45 |
+
r"\b(?:cargo)\s+check\b|"
|
| 46 |
+
r"\b(?:mvn|gradle)\s+(?:test|check|build)\b|"
|
| 47 |
+
r"\bmake\s+(?:check|test)\b|"
|
| 48 |
+
r"\b(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?(?:build|check|lint)\b|"
|
| 49 |
+
r"(?:^|[\s/])(?:bash\s+)?[^\s;&|]*test[^\s;&|]*\.sh\b|"
|
| 50 |
+
r"\bpython(?:3)?\s+-m\s+py_compile\b|"
|
| 51 |
+
r"\b(?:ruff|mypy|eslint|tsc)\b"
|
| 52 |
+
r")"
|
| 53 |
+
)
|
| 54 |
+
POSITIVE_VERIFICATION_RE = re.compile(
|
| 55 |
+
r"(?im)(?:"
|
| 56 |
+
r"^\s*OK\s*$|"
|
| 57 |
+
r"\bRan\s+\d+\s+tests?\b|"
|
| 58 |
+
r"\b\d+\s+passed\b|"
|
| 59 |
+
r"\bBUILD\s+SUCCESS(?:FUL)?\b|"
|
| 60 |
+
r"\b(?:tests?|checks?)\s+(?:passed|successful)\b|"
|
| 61 |
+
r"\b[A-Z][A-Z0-9_]+_OK\b|"
|
| 62 |
+
r"\(?(?:Bash )?completed (?:successfully )?"
|
| 63 |
+
r"(?:with no|without)(?: textual)? output\)?"
|
| 64 |
+
r")"
|
| 65 |
+
)
|
| 66 |
+
INSPECTION_COMMAND_RE = re.compile(
|
| 67 |
+
r"(?i)^\s*(?:"
|
| 68 |
+
r"cd\b[^;&|]*(?:&&|;)\s*)?"
|
| 69 |
+
r"(?:ls|pwd|find|rg|grep|cat|sed|head|tail|wc|stat|tree|git|cd)"
|
| 70 |
+
r"\b"
|
| 71 |
+
)
|
| 72 |
+
WEB_REQUEST_RE = re.compile(
|
| 73 |
+
r"(?i)\b(?:"
|
| 74 |
+
r"pesquis(?:e|ar|a)|busque|procure|not[ií]cias?|[uú]ltimas?|"
|
| 75 |
+
r"hoje|agora|atual(?:izado|izada|mente)?|search|latest|news|browser|web"
|
| 76 |
+
r")\b"
|
| 77 |
+
)
|
| 78 |
+
WEB_SUBJECT_RE = re.compile(
|
| 79 |
+
r"(?i)\b(?:"
|
| 80 |
+
r"web|internet|pesquis\w*|busc\w*|procur\w*|not[ií]cias?|"
|
| 81 |
+
r"search|latest|news|info|site|p[aá]gina"
|
| 82 |
+
r")\b"
|
| 83 |
+
)
|
| 84 |
+
LOCAL_INSPECTION_RE = re.compile(
|
| 85 |
+
r"(?i)\b(?:"
|
| 86 |
+
r"mem[oó]ria|ram|cpu|processador|disco|armazenamento|hardware|"
|
| 87 |
+
r"sistema|kernel|processos?|servi[cç]os?|rede|endere[cç]o\s+ip|"
|
| 88 |
+
r"gpu|temperatura|bateria|swap|arquivos?|diret[oó]rios?|pastas?"
|
| 89 |
+
r")\b"
|
| 90 |
+
)
|
| 91 |
+
INSPECTION_INTENT_RE = re.compile(
|
| 92 |
+
r"(?i)\b(?:"
|
| 93 |
+
r"verifi(?:que|car|ca[cç][aã]o)|confira|cheque|inspecione|"
|
| 94 |
+
r"mostre|liste|diagnostique|analise|check|inspect|show|list|explore"
|
| 95 |
+
r")\b"
|
| 96 |
+
)
|
| 97 |
+
READ_REQUEST_RE = re.compile(
|
| 98 |
+
r"(?i)\b(?:leia|ler|read|veja|ver|open|abra)\b"
|
| 99 |
+
)
|
| 100 |
+
EXPLICIT_TOOL_REQUEST_RE = re.compile(
|
| 101 |
+
r"(?i)\b(?:use|usar|utilize|utilizar|chame|chamar|call|invoke|"
|
| 102 |
+
r"execute|executar)\s+"
|
| 103 |
+
r"(?:(?:obrigatoriamente|necessariamente|somente|only|just|"
|
| 104 |
+
r"a|o|as|os|the|ferramenta|tool)\s+)*"
|
| 105 |
+
r"(?P<tool>bash|read|write|edit|glob|grep|websearch|webfetch|"
|
| 106 |
+
r"task|agent|notebookedit|lsp)\b"
|
| 107 |
+
)
|
| 108 |
+
IMPLEMENTATION_REQUEST_RE = re.compile(
|
| 109 |
+
r"(?i)\b(?:"
|
| 110 |
+
r"implemente|implement|corrija|corrigir|fix|edite|editar|modify|"
|
| 111 |
+
r"altere|alterar|crie|criar|create|write|escreva|instale|install|"
|
| 112 |
+
r"baixe|download|execute|rode|run|teste|testar|automatiz\w*"
|
| 113 |
+
r")\b"
|
| 114 |
+
)
|
| 115 |
+
PROGRAMMING_CONTEXT_RE = re.compile(
|
| 116 |
+
r"(?i)\b(?:"
|
| 117 |
+
r"arquivo|file|c[oó]digo|code|projeto|project|reposit[oó]rio|repo|"
|
| 118 |
+
r"script|programa|aplica[cç][aã]o|app|fun[cç][aã]o|function|classe|"
|
| 119 |
+
r"m[oó]dulo|module|teste|test|bug|erro|error|build|site|endpoint|"
|
| 120 |
+
r"proxy|api|depend[eê]ncia|package|solu[cç][aã]o|funcionalidade|feature"
|
| 121 |
+
r")\b"
|
| 122 |
+
)
|
| 123 |
+
ACTION_NOW_RE = re.compile(
|
| 124 |
+
r"(?i)\b(?:fa[cç]a|execute|rode|run|do)\s+(?:isso\s+)?agora\b|"
|
| 125 |
+
r"\bdo\s+it\s+now\b"
|
| 126 |
+
)
|
| 127 |
+
NO_TOOLS_RE = re.compile(
|
| 128 |
+
r"(?i)\b(?:"
|
| 129 |
+
r"n[aã]o\s+(?:use|usar|chame|chamar)|"
|
| 130 |
+
r"sem|"
|
| 131 |
+
r"do\s+not\s+(?:use|call)|"
|
| 132 |
+
r"never\s+(?:use|call)|"
|
| 133 |
+
r"without"
|
| 134 |
+
r")\s+(?:as?\s+)?(?:ferramentas?|tools?)\b"
|
| 135 |
+
)
|
| 136 |
+
SIMPLE_GREETING_RE = re.compile(
|
| 137 |
+
r"(?i)^\s*(?:oi|ol[aá]|hello|hi|hey|bom\s+dia|boa\s+tarde|boa\s+noite)"
|
| 138 |
+
r"[\s!,.?]*$"
|
| 139 |
+
)
|
| 140 |
+
OPENCLAUDE_METADATA_BLOCK_RE = re.compile(
|
| 141 |
+
r"<(?P<tag>available-deferred-tools|system-reminder)\b[^>]*>.*?</(?P=tag)>",
|
| 142 |
+
re.DOTALL | re.IGNORECASE,
|
| 143 |
+
)
|
| 144 |
+
|
| 145 |
+
|
| 146 |
+
@dataclass(frozen=True)
|
| 147 |
+
class ToolFlowState:
|
| 148 |
+
"""Request-local progress state; no conversation state is stored globally."""
|
| 149 |
+
|
| 150 |
+
active: bool = False
|
| 151 |
+
requires_tool: bool = False
|
| 152 |
+
can_finalize: bool = False
|
| 153 |
+
reason: str = ""
|
| 154 |
+
instruction: str | None = None
|
| 155 |
+
forced_tool: str | None = None
|
| 156 |
+
|
| 157 |
+
|
| 158 |
+
@dataclass(frozen=True)
|
| 159 |
+
class _ToolResultEvent:
|
| 160 |
+
name: str
|
| 161 |
+
arguments: dict[str, Any]
|
| 162 |
+
content: str
|
| 163 |
+
is_error: bool
|
| 164 |
+
batch: int
|
| 165 |
+
|
| 166 |
+
|
| 167 |
+
def _bounded_description(value: Any, limit: int) -> str:
|
| 168 |
+
"""Return a compact single-line description suitable for a model prompt."""
|
| 169 |
+
text = re.sub(r"\s+", " ", str(value or "")).strip()
|
| 170 |
+
if len(text) <= limit:
|
| 171 |
+
return text
|
| 172 |
+
shortened = text[: max(1, limit - 1)].rsplit(" ", 1)[0].rstrip()
|
| 173 |
+
return (shortened or text[: limit - 1]).rstrip() + "…"
|
| 174 |
+
|
| 175 |
+
|
| 176 |
+
def _compact_schema_descriptions(value: Any) -> Any:
|
| 177 |
+
"""Bound schema prose without removing structural validation information."""
|
| 178 |
+
if isinstance(value, Mapping):
|
| 179 |
+
return {
|
| 180 |
+
key: (
|
| 181 |
+
_bounded_description(raw_value, MAX_SCHEMA_DESCRIPTION_CHARS)
|
| 182 |
+
if key == "description"
|
| 183 |
+
else _compact_schema_descriptions(raw_value)
|
| 184 |
+
)
|
| 185 |
+
for key, raw_value in value.items()
|
| 186 |
+
}
|
| 187 |
+
if isinstance(value, list):
|
| 188 |
+
return [_compact_schema_descriptions(item) for item in value]
|
| 189 |
+
return value
|
| 190 |
+
|
| 191 |
+
|
| 192 |
+
def _content_text(content: Any) -> str:
|
| 193 |
+
if isinstance(content, str):
|
| 194 |
+
return content
|
| 195 |
+
if isinstance(content, list):
|
| 196 |
+
parts: list[str] = []
|
| 197 |
+
for block in content:
|
| 198 |
+
if isinstance(block, Mapping):
|
| 199 |
+
text = block.get("text", block.get("content", ""))
|
| 200 |
+
if text:
|
| 201 |
+
parts.append(str(text))
|
| 202 |
+
elif block is not None:
|
| 203 |
+
parts.append(str(block))
|
| 204 |
+
return "\n".join(parts)
|
| 205 |
+
return "" if content is None else str(content)
|
| 206 |
+
|
| 207 |
+
|
| 208 |
+
def _user_request_text(content: Any) -> str:
|
| 209 |
+
"""Remove OpenClaude's injected metadata before classifying user intent.
|
| 210 |
+
|
| 211 |
+
OpenClaude places deferred-tool lists, skill descriptions, and snip markers
|
| 212 |
+
inside a user-role message. Those blocks can contain words such as
|
| 213 |
+
``create``, ``code``, or ``test``; treating them as the user's request can
|
| 214 |
+
incorrectly force ``tool_choice=required`` for a plain greeting.
|
| 215 |
+
"""
|
| 216 |
+
text = _content_text(content)
|
| 217 |
+
previous = None
|
| 218 |
+
while text != previous:
|
| 219 |
+
previous = text
|
| 220 |
+
text = OPENCLAUDE_METADATA_BLOCK_RE.sub("", text)
|
| 221 |
+
return text.strip()
|
| 222 |
+
|
| 223 |
+
|
| 224 |
+
def _call_arguments(value: Any) -> dict[str, Any]:
|
| 225 |
+
if isinstance(value, Mapping):
|
| 226 |
+
return dict(value)
|
| 227 |
+
if isinstance(value, str):
|
| 228 |
+
try:
|
| 229 |
+
parsed = json.loads(value)
|
| 230 |
+
except json.JSONDecodeError:
|
| 231 |
+
return {}
|
| 232 |
+
return dict(parsed) if isinstance(parsed, Mapping) else {}
|
| 233 |
+
return {}
|
| 234 |
+
|
| 235 |
+
|
| 236 |
+
def _is_synthetic_continuation(message: Mapping[str, Any]) -> bool:
|
| 237 |
+
content = message.get("content")
|
| 238 |
+
if isinstance(content, list) and any(
|
| 239 |
+
isinstance(block, Mapping) and block.get("type") == "tool_result"
|
| 240 |
+
for block in content
|
| 241 |
+
):
|
| 242 |
+
return True
|
| 243 |
+
text = _content_text(content).casefold()
|
| 244 |
+
return (
|
| 245 |
+
not text.strip()
|
| 246 |
+
or "[tool results received]" in text
|
| 247 |
+
or (
|
| 248 |
+
"continue with the task" in text
|
| 249 |
+
and "resume your thought" in text
|
| 250 |
+
)
|
| 251 |
+
or (
|
| 252 |
+
"<system-reminder" in text
|
| 253 |
+
and not re.sub(
|
| 254 |
+
r"<system-reminder\b[^>]*>.*?</system-reminder>",
|
| 255 |
+
"",
|
| 256 |
+
text,
|
| 257 |
+
flags=re.DOTALL | re.IGNORECASE,
|
| 258 |
+
).strip()
|
| 259 |
+
)
|
| 260 |
+
)
|
| 261 |
+
|
| 262 |
+
|
| 263 |
+
def _current_turn_messages(messages: object) -> list[object]:
|
| 264 |
+
if not isinstance(messages, list):
|
| 265 |
+
return []
|
| 266 |
+
start = 0
|
| 267 |
+
for index, message in enumerate(messages):
|
| 268 |
+
if (
|
| 269 |
+
isinstance(message, Mapping)
|
| 270 |
+
and str(message.get("role", "")).casefold() == "user"
|
| 271 |
+
and not _is_synthetic_continuation(message)
|
| 272 |
+
):
|
| 273 |
+
start = index
|
| 274 |
+
return messages[start:]
|
| 275 |
+
|
| 276 |
+
|
| 277 |
+
def _tool_result_events(messages: object) -> list[_ToolResultEvent]:
|
| 278 |
+
current_messages = _current_turn_messages(messages)
|
| 279 |
+
calls_by_id: dict[str, tuple[str, dict[str, Any], int]] = {}
|
| 280 |
+
pending_order: list[str] = []
|
| 281 |
+
events: list[_ToolResultEvent] = []
|
| 282 |
+
batch = 0
|
| 283 |
+
|
| 284 |
+
for message in current_messages:
|
| 285 |
+
if not isinstance(message, Mapping):
|
| 286 |
+
continue
|
| 287 |
+
role = str(message.get("role", "")).casefold()
|
| 288 |
+
if role == "assistant":
|
| 289 |
+
raw_calls = message.get("tool_calls") or []
|
| 290 |
+
if raw_calls:
|
| 291 |
+
batch += 1
|
| 292 |
+
for index, raw_call in enumerate(raw_calls):
|
| 293 |
+
if not isinstance(raw_call, Mapping):
|
| 294 |
+
continue
|
| 295 |
+
function = raw_call.get("function")
|
| 296 |
+
if not isinstance(function, Mapping):
|
| 297 |
+
continue
|
| 298 |
+
name = function.get("name")
|
| 299 |
+
if not isinstance(name, str) or not name:
|
| 300 |
+
continue
|
| 301 |
+
call_id = raw_call.get("id")
|
| 302 |
+
if not isinstance(call_id, str) or not call_id:
|
| 303 |
+
call_id = f"__ordered_{len(calls_by_id)}_{index}"
|
| 304 |
+
calls_by_id[call_id] = (
|
| 305 |
+
name,
|
| 306 |
+
_call_arguments(function.get("arguments", {})),
|
| 307 |
+
batch,
|
| 308 |
+
)
|
| 309 |
+
pending_order.append(call_id)
|
| 310 |
+
continue
|
| 311 |
+
if role != "tool":
|
| 312 |
+
continue
|
| 313 |
+
|
| 314 |
+
call_id = message.get("tool_call_id")
|
| 315 |
+
call: tuple[str, dict[str, Any], int] | None = None
|
| 316 |
+
if isinstance(call_id, str) and call_id:
|
| 317 |
+
call = calls_by_id.pop(call_id, None)
|
| 318 |
+
if call_id in pending_order:
|
| 319 |
+
pending_order.remove(call_id)
|
| 320 |
+
elif pending_order:
|
| 321 |
+
fallback_id = pending_order.pop(0)
|
| 322 |
+
call = calls_by_id.pop(fallback_id, None)
|
| 323 |
+
|
| 324 |
+
if call is None:
|
| 325 |
+
explicit_name = message.get("name")
|
| 326 |
+
if not isinstance(explicit_name, str) or not explicit_name:
|
| 327 |
+
continue
|
| 328 |
+
call = (explicit_name, {}, batch)
|
| 329 |
+
|
| 330 |
+
content = _content_text(message.get("content"))
|
| 331 |
+
structured_error = message.get("is_error") is True
|
| 332 |
+
if isinstance(message.get("content"), list):
|
| 333 |
+
structured_error = structured_error or any(
|
| 334 |
+
isinstance(block, Mapping) and block.get("is_error") is True
|
| 335 |
+
for block in message["content"]
|
| 336 |
+
)
|
| 337 |
+
events.append(
|
| 338 |
+
_ToolResultEvent(
|
| 339 |
+
name=call[0],
|
| 340 |
+
arguments=call[1],
|
| 341 |
+
content=content,
|
| 342 |
+
is_error=structured_error or bool(FAILED_RESULT_RE.search(content)),
|
| 343 |
+
batch=call[2],
|
| 344 |
+
)
|
| 345 |
+
)
|
| 346 |
+
return events
|
| 347 |
+
|
| 348 |
+
|
| 349 |
+
def _bash_command(event: _ToolResultEvent) -> str:
|
| 350 |
+
command = event.arguments.get("command", event.arguments.get("cmd", ""))
|
| 351 |
+
return command if isinstance(command, str) else str(command)
|
| 352 |
+
|
| 353 |
+
|
| 354 |
+
def _bash_proves_completion(event: _ToolResultEvent) -> bool:
|
| 355 |
+
if event.is_error:
|
| 356 |
+
return False
|
| 357 |
+
command = _bash_command(event)
|
| 358 |
+
if not VERIFICATION_COMMAND_RE.search(command):
|
| 359 |
+
return False
|
| 360 |
+
return bool(POSITIVE_VERIFICATION_RE.search(event.content))
|
| 361 |
+
|
| 362 |
+
|
| 363 |
+
def _latest_user_request(messages: object) -> str:
|
| 364 |
+
requests: list[str] = []
|
| 365 |
+
if not isinstance(messages, list):
|
| 366 |
+
return ""
|
| 367 |
+
for message in messages:
|
| 368 |
+
if (
|
| 369 |
+
isinstance(message, Mapping)
|
| 370 |
+
and str(message.get("role", "")).casefold() == "user"
|
| 371 |
+
and not _is_synthetic_continuation(message)
|
| 372 |
+
):
|
| 373 |
+
text = _user_request_text(message.get("content"))
|
| 374 |
+
if text:
|
| 375 |
+
requests.append(text)
|
| 376 |
+
if not requests:
|
| 377 |
+
return ""
|
| 378 |
+
latest = requests[-1]
|
| 379 |
+
if len(requests) > 1 and ACTION_NOW_RE.search(latest):
|
| 380 |
+
return requests[-2] + "\n" + latest
|
| 381 |
+
return latest
|
| 382 |
+
|
| 383 |
+
|
| 384 |
+
def is_simple_greeting(messages: object) -> bool:
|
| 385 |
+
"""Identify a greeting that does not need a model or tool prompt.
|
| 386 |
+
|
| 387 |
+
OpenClaude sends its complete tool catalog even for ``ola``. Calling a
|
| 388 |
+
model on ZeroGPU for that turn adds unnecessary queue time, so the API can
|
| 389 |
+
answer it deterministically before inference.
|
| 390 |
+
"""
|
| 391 |
+
return bool(SIMPLE_GREETING_RE.fullmatch(_latest_user_request(messages)))
|
| 392 |
+
|
| 393 |
+
|
| 394 |
+
def _explicitly_disables_tools(messages: object) -> bool:
|
| 395 |
+
if not isinstance(messages, list):
|
| 396 |
+
return False
|
| 397 |
+
return any(
|
| 398 |
+
isinstance(message, Mapping)
|
| 399 |
+
and str(message.get("role", "")).casefold()
|
| 400 |
+
in {"system", "developer", "user"}
|
| 401 |
+
and bool(NO_TOOLS_RE.search(_content_text(message.get("content"))))
|
| 402 |
+
for message in messages
|
| 403 |
+
)
|
| 404 |
+
|
| 405 |
+
|
| 406 |
+
def _initial_tool_flow(
|
| 407 |
+
messages: object,
|
| 408 |
+
available_by_fold: Mapping[str, str],
|
| 409 |
+
) -> ToolFlowState:
|
| 410 |
+
"""Force action for concrete first-turn requests instead of accepting plans."""
|
| 411 |
+
request = _latest_user_request(messages)
|
| 412 |
+
if not request or not available_by_fold:
|
| 413 |
+
return ToolFlowState()
|
| 414 |
+
|
| 415 |
+
explicit_tool = EXPLICIT_TOOL_REQUEST_RE.search(request)
|
| 416 |
+
if explicit_tool:
|
| 417 |
+
requested_name = explicit_tool.group("tool").casefold()
|
| 418 |
+
forced_tool = available_by_fold.get(requested_name)
|
| 419 |
+
if forced_tool is None:
|
| 420 |
+
forced_tool = available_by_fold.get(
|
| 421 |
+
{"agent": "task", "task": "agent"}.get(requested_name, "")
|
| 422 |
+
)
|
| 423 |
+
if forced_tool is not None:
|
| 424 |
+
return ToolFlowState(
|
| 425 |
+
active=True,
|
| 426 |
+
requires_tool=True,
|
| 427 |
+
reason=f"the user explicitly requested the {forced_tool} tool",
|
| 428 |
+
instruction=(
|
| 429 |
+
f"OPENCLAUDE FLOW STATE: call {forced_tool} now because the "
|
| 430 |
+
"user explicitly requested it. Do not print a sample call "
|
| 431 |
+
"as prose and do not answer with a plan."
|
| 432 |
+
),
|
| 433 |
+
forced_tool=forced_tool,
|
| 434 |
+
)
|
| 435 |
+
|
| 436 |
+
if (
|
| 437 |
+
"websearch" in available_by_fold
|
| 438 |
+
and WEB_REQUEST_RE.search(request)
|
| 439 |
+
and WEB_SUBJECT_RE.search(request)
|
| 440 |
+
):
|
| 441 |
+
return ToolFlowState(
|
| 442 |
+
active=True,
|
| 443 |
+
requires_tool=True,
|
| 444 |
+
reason="the user requested current web research",
|
| 445 |
+
instruction=(
|
| 446 |
+
"OPENCLAUDE FLOW STATE: perform the requested research now. "
|
| 447 |
+
"Call WebSearch with a concise query; do not merely describe how "
|
| 448 |
+
"you would search and do not substitute curl or invented APIs."
|
| 449 |
+
),
|
| 450 |
+
forced_tool=available_by_fold["websearch"],
|
| 451 |
+
)
|
| 452 |
+
|
| 453 |
+
if (
|
| 454 |
+
"bash" in available_by_fold
|
| 455 |
+
and LOCAL_INSPECTION_RE.search(request)
|
| 456 |
+
and INSPECTION_INTENT_RE.search(request)
|
| 457 |
+
):
|
| 458 |
+
return ToolFlowState(
|
| 459 |
+
active=True,
|
| 460 |
+
requires_tool=True,
|
| 461 |
+
reason="the user requested inspection of the local system",
|
| 462 |
+
instruction=(
|
| 463 |
+
"OPENCLAUDE FLOW STATE: inspect the local system now. Call Bash "
|
| 464 |
+
"with a safe read-only command that directly answers the request; "
|
| 465 |
+
"do not print a command as prose and do not ask for confirmation."
|
| 466 |
+
),
|
| 467 |
+
forced_tool=available_by_fold["bash"],
|
| 468 |
+
)
|
| 469 |
+
|
| 470 |
+
if "read" in available_by_fold and READ_REQUEST_RE.search(request):
|
| 471 |
+
return ToolFlowState(
|
| 472 |
+
active=True,
|
| 473 |
+
requires_tool=True,
|
| 474 |
+
reason="the user explicitly requested reading a file",
|
| 475 |
+
instruction=(
|
| 476 |
+
"OPENCLAUDE FLOW STATE: call Read now for the relevant file. "
|
| 477 |
+
"Do not describe a future read operation."
|
| 478 |
+
),
|
| 479 |
+
forced_tool=available_by_fold["read"],
|
| 480 |
+
)
|
| 481 |
+
|
| 482 |
+
concrete_implementation = bool(
|
| 483 |
+
IMPLEMENTATION_REQUEST_RE.search(request)
|
| 484 |
+
and (
|
| 485 |
+
PROGRAMMING_CONTEXT_RE.search(request)
|
| 486 |
+
or re.search(r"(?i)\bautomatiz\w*\b", request)
|
| 487 |
+
)
|
| 488 |
+
)
|
| 489 |
+
if ACTION_NOW_RE.search(request) or concrete_implementation:
|
| 490 |
+
return ToolFlowState(
|
| 491 |
+
active=True,
|
| 492 |
+
requires_tool=True,
|
| 493 |
+
reason="the user requested immediate tool-backed action",
|
| 494 |
+
instruction=(
|
| 495 |
+
"OPENCLAUDE FLOW STATE: act on the request now by calling one "
|
| 496 |
+
"appropriate available tool. Do not answer with a plan, example "
|
| 497 |
+
"commands, or a request for the user to repeat the task."
|
| 498 |
+
),
|
| 499 |
+
)
|
| 500 |
+
|
| 501 |
+
# OpenClaude may send ``tool_choice=required`` even for greetings and
|
| 502 |
+
# other conversational turns. Those turns must be allowed to finalize;
|
| 503 |
+
# requiring a synthetic tool call makes a harmless "oi" become a 502.
|
| 504 |
+
return ToolFlowState(reason="no concrete tool action was requested")
|
| 505 |
+
|
| 506 |
+
|
| 507 |
+
def analyze_tool_flow(
|
| 508 |
+
messages: object,
|
| 509 |
+
raw_tools: object,
|
| 510 |
+
) -> ToolFlowState:
|
| 511 |
+
"""Derive whether an agent must continue or may emit its final response."""
|
| 512 |
+
if _explicitly_disables_tools(messages):
|
| 513 |
+
return ToolFlowState(
|
| 514 |
+
can_finalize=True,
|
| 515 |
+
reason="the request explicitly disables all tools",
|
| 516 |
+
)
|
| 517 |
+
available_by_fold = {
|
| 518 |
+
tool["function"]["name"].casefold(): tool["function"]["name"]
|
| 519 |
+
for tool in normalize_tools(raw_tools)
|
| 520 |
+
}
|
| 521 |
+
available = set(available_by_fold)
|
| 522 |
+
events = _tool_result_events(messages)
|
| 523 |
+
if not events:
|
| 524 |
+
return _initial_tool_flow(messages, available_by_fold)
|
| 525 |
+
|
| 526 |
+
# A successful search/fetch is terminal evidence for a research request.
|
| 527 |
+
# This intentionally prevents WebSearch -> WebFetch -> repeated curl loops.
|
| 528 |
+
web_evidence = any(
|
| 529 |
+
event.name.casefold() in {"websearch", "webfetch"}
|
| 530 |
+
and not event.is_error
|
| 531 |
+
and bool(event.content.strip())
|
| 532 |
+
for event in events
|
| 533 |
+
)
|
| 534 |
+
|
| 535 |
+
request = _latest_user_request(messages)
|
| 536 |
+
agentic_intent = not request or bool(
|
| 537 |
+
IMPLEMENTATION_REQUEST_RE.search(request)
|
| 538 |
+
and (
|
| 539 |
+
PROGRAMMING_CONTEXT_RE.search(request)
|
| 540 |
+
or re.search(r"(?i)\bautomatiz\w*\b", request)
|
| 541 |
+
)
|
| 542 |
+
)
|
| 543 |
+
agentic = (
|
| 544 |
+
agentic_intent
|
| 545 |
+
and "bash" in available
|
| 546 |
+
and bool({"edit", "write"} & available)
|
| 547 |
+
)
|
| 548 |
+
dirty = False
|
| 549 |
+
dirty_batch = -1
|
| 550 |
+
agentic_started = False
|
| 551 |
+
last_reason = ""
|
| 552 |
+
|
| 553 |
+
if agentic:
|
| 554 |
+
for event in events:
|
| 555 |
+
name = event.name.casefold()
|
| 556 |
+
if name == "read":
|
| 557 |
+
agentic_started = True
|
| 558 |
+
dirty = True
|
| 559 |
+
dirty_batch = max(dirty_batch, event.batch)
|
| 560 |
+
last_reason = "files were inspected but implementation is still pending"
|
| 561 |
+
elif name in {"edit", "write"}:
|
| 562 |
+
agentic_started = True
|
| 563 |
+
dirty = True
|
| 564 |
+
dirty_batch = max(dirty_batch, event.batch)
|
| 565 |
+
last_reason = "files changed and must be verified with Bash"
|
| 566 |
+
elif event.is_error and agentic_started:
|
| 567 |
+
dirty = True
|
| 568 |
+
dirty_batch = max(dirty_batch, event.batch)
|
| 569 |
+
last_reason = f"{event.name} returned an error that must be recovered"
|
| 570 |
+
elif name == "bash":
|
| 571 |
+
command = _bash_command(event)
|
| 572 |
+
if event.is_error:
|
| 573 |
+
agentic_started = True
|
| 574 |
+
dirty = True
|
| 575 |
+
dirty_batch = max(dirty_batch, event.batch)
|
| 576 |
+
last_reason = "the Bash command or test failed"
|
| 577 |
+
elif INSPECTION_COMMAND_RE.search(command):
|
| 578 |
+
agentic_started = True
|
| 579 |
+
dirty = True
|
| 580 |
+
dirty_batch = max(dirty_batch, event.batch)
|
| 581 |
+
last_reason = "inspection output is not completion evidence"
|
| 582 |
+
elif (
|
| 583 |
+
agentic_started
|
| 584 |
+
and dirty
|
| 585 |
+
and event.batch > dirty_batch
|
| 586 |
+
and _bash_proves_completion(event)
|
| 587 |
+
):
|
| 588 |
+
dirty = False
|
| 589 |
+
last_reason = "a Bash verification passed after the latest change"
|
| 590 |
+
elif agentic_started and dirty:
|
| 591 |
+
last_reason = "Bash did not provide positive test evidence"
|
| 592 |
+
|
| 593 |
+
if agentic_started and dirty:
|
| 594 |
+
return ToolFlowState(
|
| 595 |
+
active=True,
|
| 596 |
+
requires_tool=True,
|
| 597 |
+
reason=last_reason,
|
| 598 |
+
instruction=(
|
| 599 |
+
"OPENCLAUDE FLOW STATE: the task is not complete. "
|
| 600 |
+
f"Reason: {last_reason}. Call exactly one appropriate tool now; "
|
| 601 |
+
"do not describe a future plan. After reading, edit or write the "
|
| 602 |
+
"implementation. After changes, use Bash to run the requested "
|
| 603 |
+
"tests and continue fixing failures until the output proves success."
|
| 604 |
+
),
|
| 605 |
+
)
|
| 606 |
+
|
| 607 |
+
if agentic_started and not dirty:
|
| 608 |
+
return ToolFlowState(
|
| 609 |
+
active=True,
|
| 610 |
+
can_finalize=True,
|
| 611 |
+
reason=last_reason,
|
| 612 |
+
instruction=(
|
| 613 |
+
"OPENCLAUDE FLOW STATE: verification passed after the latest "
|
| 614 |
+
"change. Do not call another tool. Report the completed work and "
|
| 615 |
+
"the test evidence directly in Brazilian Portuguese."
|
| 616 |
+
),
|
| 617 |
+
)
|
| 618 |
+
|
| 619 |
+
if web_evidence:
|
| 620 |
+
return ToolFlowState(
|
| 621 |
+
active=True,
|
| 622 |
+
can_finalize=True,
|
| 623 |
+
reason="usable web evidence is available",
|
| 624 |
+
instruction=(
|
| 625 |
+
"OPENCLAUDE FLOW STATE: usable WebSearch/WebFetch results are "
|
| 626 |
+
"already available. Do not call WebFetch, Bash, curl, or another "
|
| 627 |
+
"tool. Synthesize a concrete answer now from the supplied results, "
|
| 628 |
+
"include useful source links, and never invent API keys or facts."
|
| 629 |
+
),
|
| 630 |
+
)
|
| 631 |
+
|
| 632 |
+
last_webfetch_error = max(
|
| 633 |
+
(
|
| 634 |
+
index
|
| 635 |
+
for index, event in enumerate(events)
|
| 636 |
+
if event.name.casefold() == "webfetch" and event.is_error
|
| 637 |
+
),
|
| 638 |
+
default=-1,
|
| 639 |
+
)
|
| 640 |
+
last_websearch_error = max(
|
| 641 |
+
(
|
| 642 |
+
index
|
| 643 |
+
for index, event in enumerate(events)
|
| 644 |
+
if event.name.casefold() == "websearch" and event.is_error
|
| 645 |
+
),
|
| 646 |
+
default=-1,
|
| 647 |
+
)
|
| 648 |
+
toolsearch_recovered = (
|
| 649 |
+
last_webfetch_error >= 0
|
| 650 |
+
and any(
|
| 651 |
+
index > last_webfetch_error
|
| 652 |
+
and event.name.casefold() == "toolsearch"
|
| 653 |
+
and not event.is_error
|
| 654 |
+
for index, event in enumerate(events)
|
| 655 |
+
)
|
| 656 |
+
)
|
| 657 |
+
|
| 658 |
+
forced_tool: str | None = None
|
| 659 |
+
recovery = ""
|
| 660 |
+
web_error_name = ""
|
| 661 |
+
if last_webfetch_error >= 0:
|
| 662 |
+
web_error_name = "WebFetch"
|
| 663 |
+
if toolsearch_recovered and "webfetch" in available:
|
| 664 |
+
forced_tool = available_by_fold["webfetch"]
|
| 665 |
+
recovery = (
|
| 666 |
+
"Retry WebFetch now with both required fields: url and prompt."
|
| 667 |
+
)
|
| 668 |
+
elif "webfetch" not in available and "toolsearch" in available:
|
| 669 |
+
forced_tool = available_by_fold["toolsearch"]
|
| 670 |
+
recovery = (
|
| 671 |
+
"Load WebFetch by calling ToolSearch with query select:WebFetch."
|
| 672 |
+
)
|
| 673 |
+
elif "webfetch" in available:
|
| 674 |
+
forced_tool = available_by_fold["webfetch"]
|
| 675 |
+
recovery = (
|
| 676 |
+
"Retry WebFetch with both required fields: url and prompt."
|
| 677 |
+
)
|
| 678 |
+
elif "websearch" in available:
|
| 679 |
+
forced_tool = available_by_fold["websearch"]
|
| 680 |
+
recovery = "Recover with WebSearch using a concise, relevant query."
|
| 681 |
+
elif last_websearch_error >= 0 and "websearch" in available:
|
| 682 |
+
web_error_name = "WebSearch"
|
| 683 |
+
forced_tool = available_by_fold["websearch"]
|
| 684 |
+
recovery = "Retry WebSearch using a concise, relevant query."
|
| 685 |
+
|
| 686 |
+
if forced_tool:
|
| 687 |
+
return ToolFlowState(
|
| 688 |
+
active=True,
|
| 689 |
+
requires_tool=True,
|
| 690 |
+
reason=f"{web_error_name} returned an error",
|
| 691 |
+
instruction=(
|
| 692 |
+
f"OPENCLAUDE FLOW STATE: {web_error_name} failed. "
|
| 693 |
+
f"{recovery} Do not answer with a plan and do not invent "
|
| 694 |
+
"credentials, endpoints, or placeholder tokens."
|
| 695 |
+
),
|
| 696 |
+
forced_tool=forced_tool,
|
| 697 |
+
)
|
| 698 |
+
|
| 699 |
+
# Read already provides the requested evidence. Mark it terminal so
|
| 700 |
+
# OpenClaude's repeated ``tool_choice=required`` does not make a small
|
| 701 |
+
# model call Read forever. Keep generic Bash inspection neutral: the
|
| 702 |
+
# existing flow still lets the model decide how to summarize it.
|
| 703 |
+
last_event = events[-1]
|
| 704 |
+
if (
|
| 705 |
+
last_event.name.casefold() == "read"
|
| 706 |
+
and not last_event.is_error
|
| 707 |
+
and bool(last_event.content.strip())
|
| 708 |
+
):
|
| 709 |
+
return ToolFlowState(
|
| 710 |
+
active=True,
|
| 711 |
+
can_finalize=True,
|
| 712 |
+
reason="a successful Read result is available",
|
| 713 |
+
instruction=(
|
| 714 |
+
"OPENCLAUDE FLOW STATE: the requested Read tool returned usable "
|
| 715 |
+
"evidence. Do not call another tool; synthesize the answer "
|
| 716 |
+
"directly from the result in Brazilian Portuguese."
|
| 717 |
+
),
|
| 718 |
+
)
|
| 719 |
+
|
| 720 |
+
return ToolFlowState()
|
| 721 |
+
|
| 722 |
+
|
| 723 |
+
def resolve_tool_choice(
|
| 724 |
+
requested_choice: object,
|
| 725 |
+
state: ToolFlowState,
|
| 726 |
+
) -> object:
|
| 727 |
+
"""Resolve client tool choice against the reconstructed conversation state.
|
| 728 |
+
|
| 729 |
+
A concrete function choice remains authoritative. ``required`` is slightly
|
| 730 |
+
different for OpenClaude: the client repeats it across turns, so the Space
|
| 731 |
+
must narrow it to a detected function, or downgrade it to ``none`` once a
|
| 732 |
+
usable tool result is already available / no concrete tool action exists.
|
| 733 |
+
"""
|
| 734 |
+
if isinstance(requested_choice, Mapping):
|
| 735 |
+
return requested_choice
|
| 736 |
+
|
| 737 |
+
requested_mode = (
|
| 738 |
+
requested_choice.casefold()
|
| 739 |
+
if isinstance(requested_choice, str)
|
| 740 |
+
else None
|
| 741 |
+
)
|
| 742 |
+
|
| 743 |
+
if requested_mode == "required":
|
| 744 |
+
if state.requires_tool:
|
| 745 |
+
if state.forced_tool:
|
| 746 |
+
return {
|
| 747 |
+
"type": "function",
|
| 748 |
+
"function": {"name": state.forced_tool},
|
| 749 |
+
}
|
| 750 |
+
return "required"
|
| 751 |
+
if state.can_finalize or state.reason == "no concrete tool action was requested":
|
| 752 |
+
return "none"
|
| 753 |
+
return "required"
|
| 754 |
+
|
| 755 |
+
is_auto = requested_choice is None or requested_mode == "auto"
|
| 756 |
+
if not is_auto:
|
| 757 |
+
return requested_choice
|
| 758 |
+
|
| 759 |
+
if state.can_finalize or state.reason == "no concrete tool action was requested":
|
| 760 |
+
return "none"
|
| 761 |
+
if state.requires_tool:
|
| 762 |
+
if state.forced_tool:
|
| 763 |
+
return {
|
| 764 |
+
"type": "function",
|
| 765 |
+
"function": {"name": state.forced_tool},
|
| 766 |
+
}
|
| 767 |
+
return "required"
|
| 768 |
+
return requested_choice
|
| 769 |
+
|
| 770 |
+
|
| 771 |
+
def normalize_tools(raw_tools: object) -> list[dict[str, Any]]:
|
| 772 |
+
"""Return valid function definitions for Qwen's native tool template."""
|
| 773 |
+
if not isinstance(raw_tools, list):
|
| 774 |
+
return []
|
| 775 |
+
|
| 776 |
+
normalized: list[dict[str, Any]] = []
|
| 777 |
+
for raw_tool in raw_tools:
|
| 778 |
+
if not isinstance(raw_tool, Mapping):
|
| 779 |
+
continue
|
| 780 |
+
function = raw_tool.get("function")
|
| 781 |
+
candidate = function if isinstance(function, Mapping) else raw_tool
|
| 782 |
+
name = candidate.get("name")
|
| 783 |
+
if not isinstance(name, str) or not name:
|
| 784 |
+
continue
|
| 785 |
+
parameters = candidate.get(
|
| 786 |
+
"parameters", candidate.get("input_schema", EMPTY_PARAMETERS)
|
| 787 |
+
)
|
| 788 |
+
if not isinstance(parameters, Mapping):
|
| 789 |
+
parameters = EMPTY_PARAMETERS
|
| 790 |
+
normalized.append(
|
| 791 |
+
{
|
| 792 |
+
"type": "function",
|
| 793 |
+
"function": {
|
| 794 |
+
"name": name,
|
| 795 |
+
"description": _bounded_description(
|
| 796 |
+
candidate.get("description"), MAX_TOOL_DESCRIPTION_CHARS
|
| 797 |
+
),
|
| 798 |
+
"parameters": _compact_schema_descriptions(parameters),
|
| 799 |
+
},
|
| 800 |
+
}
|
| 801 |
+
)
|
| 802 |
+
return normalized
|
| 803 |
+
|
| 804 |
+
|
| 805 |
+
def select_tools(
|
| 806 |
+
raw_tools: object,
|
| 807 |
+
tool_choice: object,
|
| 808 |
+
) -> tuple[list[dict[str, Any]], str]:
|
| 809 |
+
"""Apply OpenAI ``tool_choice`` semantics before prompting the model.
|
| 810 |
+
|
| 811 |
+
The returned mode is one of ``auto``, ``none``, ``required``, or
|
| 812 |
+
``forced``. A forced choice only exposes the selected function to Qwen,
|
| 813 |
+
which is the most reliable way to enforce it with a native tool template.
|
| 814 |
+
"""
|
| 815 |
+
tools = normalize_tools(raw_tools)
|
| 816 |
+
if tool_choice is None:
|
| 817 |
+
return tools, "auto"
|
| 818 |
+
|
| 819 |
+
if isinstance(tool_choice, str):
|
| 820 |
+
mode = tool_choice.casefold()
|
| 821 |
+
if mode == "none":
|
| 822 |
+
return [], "none"
|
| 823 |
+
if mode in {"auto", "required"}:
|
| 824 |
+
if mode == "required" and not tools:
|
| 825 |
+
raise ValueError("tool_choice='required' needs at least one tool")
|
| 826 |
+
return tools, mode
|
| 827 |
+
raise ValueError(f"Unsupported tool_choice: {tool_choice}")
|
| 828 |
+
|
| 829 |
+
if not isinstance(tool_choice, Mapping):
|
| 830 |
+
raise ValueError("tool_choice must be 'auto', 'none', 'required', or a function")
|
| 831 |
+
function = tool_choice.get("function")
|
| 832 |
+
name = function.get("name") if isinstance(function, Mapping) else None
|
| 833 |
+
if tool_choice.get("type") != "function" or not isinstance(name, str) or not name:
|
| 834 |
+
raise ValueError("Forced tool_choice must contain function.name")
|
| 835 |
+
|
| 836 |
+
selected = [
|
| 837 |
+
tool
|
| 838 |
+
for tool in tools
|
| 839 |
+
if tool["function"]["name"].casefold() == name.casefold()
|
| 840 |
+
]
|
| 841 |
+
if not selected:
|
| 842 |
+
raise ValueError(f"Forced tool is not defined in tools: {name}")
|
| 843 |
+
return selected[:1], "forced"
|
| 844 |
+
|
| 845 |
+
|
| 846 |
+
def tool_names(tools: list[dict[str, Any]]) -> set[str]:
|
| 847 |
+
return {tool["function"]["name"] for tool in tools}
|
| 848 |
+
|
| 849 |
+
|
| 850 |
+
def indexed_tool_calls(calls: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
| 851 |
+
"""Add the per-call index required in streamed OpenAI deltas."""
|
| 852 |
+
return [{**call, "index": index} for index, call in enumerate(calls)]
|
| 853 |
+
|
| 854 |
+
|
| 855 |
+
def tool_choice_instruction(mode: str, tools: list[dict[str, Any]]) -> str | None:
|
| 856 |
+
"""Supply the constraint that Qwen's template cannot express directly."""
|
| 857 |
+
if mode == "required":
|
| 858 |
+
return "You must call one or more of the available tools in this response."
|
| 859 |
+
if mode == "forced":
|
| 860 |
+
return (
|
| 861 |
+
f"You must call the {tools[0]['function']['name']} tool in this response. "
|
| 862 |
+
"Do not answer with plain text."
|
| 863 |
+
)
|
| 864 |
+
return None
|
| 865 |
+
|
| 866 |
+
|
| 867 |
+
def _schema_example(parameters: object) -> dict[str, Any]:
|
| 868 |
+
if not isinstance(parameters, Mapping):
|
| 869 |
+
return {}
|
| 870 |
+
properties = parameters.get("properties")
|
| 871 |
+
if not isinstance(properties, Mapping):
|
| 872 |
+
return {}
|
| 873 |
+
required = parameters.get("required")
|
| 874 |
+
keys = required if isinstance(required, list) and required else list(properties)[:1]
|
| 875 |
+
example: dict[str, Any] = {}
|
| 876 |
+
for key in keys:
|
| 877 |
+
if not isinstance(key, str):
|
| 878 |
+
continue
|
| 879 |
+
raw_schema = properties.get(key)
|
| 880 |
+
schema = raw_schema if isinstance(raw_schema, Mapping) else {}
|
| 881 |
+
value_type = schema.get("type")
|
| 882 |
+
if value_type in {"integer", "number"}:
|
| 883 |
+
value: Any = 1
|
| 884 |
+
elif value_type == "boolean":
|
| 885 |
+
value = True
|
| 886 |
+
elif value_type == "array":
|
| 887 |
+
value = []
|
| 888 |
+
elif value_type == "object":
|
| 889 |
+
value = {}
|
| 890 |
+
elif "path" in key.casefold():
|
| 891 |
+
value = "/absolute/path"
|
| 892 |
+
elif "query" in key.casefold():
|
| 893 |
+
value = "search terms"
|
| 894 |
+
elif key.casefold() == "url":
|
| 895 |
+
value = "https://example.com"
|
| 896 |
+
else:
|
| 897 |
+
value = "value"
|
| 898 |
+
example[key] = value
|
| 899 |
+
return example
|
| 900 |
+
|
| 901 |
+
|
| 902 |
+
def tool_protocol_instruction(
|
| 903 |
+
tools: list[dict[str, Any]],
|
| 904 |
+
parallel_tool_calls: bool = False,
|
| 905 |
+
) -> str | None:
|
| 906 |
+
"""Return the complete notebook-agent contract enforced by the Space."""
|
| 907 |
+
if not tools:
|
| 908 |
+
return None
|
| 909 |
+
lines = [
|
| 910 |
+
"OPENAI TOOL CALL FORMAT — MANDATORY",
|
| 911 |
+
"You are operating on the user's real notebook, not a simulation.",
|
| 912 |
+
"Always communicate with the user in Brazilian Portuguese (pt-BR).",
|
| 913 |
+
"Perform requested implementation, diagnosis, download, execution, "
|
| 914 |
+
"testing, local inspection, or current web research with the available "
|
| 915 |
+
"tools instead of describing commands or a future plan.",
|
| 916 |
+
"Never claim that a file changed, a command ran, or a test passed unless "
|
| 917 |
+
"a tool result in this conversation proves it.",
|
| 918 |
+
"After WebSearch or WebFetch returns usable evidence, synthesize the "
|
| 919 |
+
"answer from it. Do not fall back to repeated curl calls.",
|
| 920 |
+
"Never invent API keys, tokens, endpoints, or placeholder credentials.",
|
| 921 |
+
"For greetings, small talk, or a self-contained factual answer, respond "
|
| 922 |
+
"directly without a tool unless the flow state below requires one.",
|
| 923 |
+
(
|
| 924 |
+
"When calling tools, emit one or more complete tool calls and no prose, "
|
| 925 |
+
"Markdown, or code fence. Multiple calls are allowed only when they are "
|
| 926 |
+
"independent and can run in parallel."
|
| 927 |
+
if parallel_tool_calls
|
| 928 |
+
else "When calling a tool, emit exactly one call and no prose, Markdown, "
|
| 929 |
+
"or code fence."
|
| 930 |
+
),
|
| 931 |
+
'Exact syntax: <tool_call>{"name":"TOOL_NAME","arguments":{"key":"value"}}</tool_call>',
|
| 932 |
+
"Arguments must be valid JSON matching the selected schema.",
|
| 933 |
+
"Available tools:",
|
| 934 |
+
]
|
| 935 |
+
available_names = {
|
| 936 |
+
str(tool.get("function", {}).get("name", "")).casefold()
|
| 937 |
+
for tool in tools
|
| 938 |
+
if isinstance(tool.get("function"), Mapping)
|
| 939 |
+
}
|
| 940 |
+
if "webfetch" in available_names:
|
| 941 |
+
lines.insert(
|
| 942 |
+
5,
|
| 943 |
+
"Call only a tool listed below. Follow every tool schema exactly. "
|
| 944 |
+
"WebFetch requires both url and prompt; never omit required fields.",
|
| 945 |
+
)
|
| 946 |
+
else:
|
| 947 |
+
lines.insert(
|
| 948 |
+
5,
|
| 949 |
+
"Call only a tool listed below. Deferred tools are unavailable in "
|
| 950 |
+
"this backend; explain when a needed capability is not listed "
|
| 951 |
+
"instead of invoking an unlisted tool.",
|
| 952 |
+
)
|
| 953 |
+
first_example: tuple[str, dict[str, Any]] | None = None
|
| 954 |
+
for tool in tools:
|
| 955 |
+
function = tool.get("function")
|
| 956 |
+
if not isinstance(function, Mapping):
|
| 957 |
+
continue
|
| 958 |
+
name = function.get("name")
|
| 959 |
+
if not isinstance(name, str) or not name:
|
| 960 |
+
continue
|
| 961 |
+
parameters = function.get("parameters")
|
| 962 |
+
lines.append(
|
| 963 |
+
json.dumps(
|
| 964 |
+
{
|
| 965 |
+
"name": name,
|
| 966 |
+
"description": str(function.get("description") or ""),
|
| 967 |
+
"parameters": (
|
| 968 |
+
dict(parameters)
|
| 969 |
+
if isinstance(parameters, Mapping)
|
| 970 |
+
else EMPTY_PARAMETERS
|
| 971 |
+
),
|
| 972 |
+
},
|
| 973 |
+
ensure_ascii=False,
|
| 974 |
+
separators=(",", ":"),
|
| 975 |
+
)
|
| 976 |
+
)
|
| 977 |
+
if first_example is None:
|
| 978 |
+
first_example = (name, _schema_example(parameters))
|
| 979 |
+
if first_example:
|
| 980 |
+
lines.append(
|
| 981 |
+
"Example syntax: <tool_call>"
|
| 982 |
+
+ json.dumps(
|
| 983 |
+
{
|
| 984 |
+
"name": first_example[0],
|
| 985 |
+
"arguments": first_example[1],
|
| 986 |
+
},
|
| 987 |
+
ensure_ascii=False,
|
| 988 |
+
separators=(",", ":"),
|
| 989 |
+
)
|
| 990 |
+
+ "</tool_call>"
|
| 991 |
+
)
|
| 992 |
+
return "\n".join(lines)
|
| 993 |
+
|
| 994 |
+
|
| 995 |
+
def text_content(content: Any) -> str:
|
| 996 |
+
"""Convert text-only OpenAI message blocks into chat-template text."""
|
| 997 |
+
if isinstance(content, str):
|
| 998 |
+
return content
|
| 999 |
+
if isinstance(content, list):
|
| 1000 |
+
return "\n".join(
|
| 1001 |
+
block.get("text", "")
|
| 1002 |
+
for block in content
|
| 1003 |
+
if isinstance(block, Mapping)
|
| 1004 |
+
and block.get("type") in {"text", "input_text"}
|
| 1005 |
+
)
|
| 1006 |
+
return "" if content is None else str(content)
|
| 1007 |
+
|
| 1008 |
+
|
| 1009 |
+
def normalized_tool_calls(raw_calls: object) -> list[dict[str, Any]]:
|
| 1010 |
+
"""Keep valid OpenAI calls in the shape Qwen's template understands."""
|
| 1011 |
+
if not isinstance(raw_calls, list):
|
| 1012 |
+
return []
|
| 1013 |
+
|
| 1014 |
+
calls: list[dict[str, Any]] = []
|
| 1015 |
+
for raw_call in raw_calls:
|
| 1016 |
+
if not isinstance(raw_call, Mapping):
|
| 1017 |
+
continue
|
| 1018 |
+
function = raw_call.get("function")
|
| 1019 |
+
if not isinstance(function, Mapping):
|
| 1020 |
+
continue
|
| 1021 |
+
name = function.get("name")
|
| 1022 |
+
if not isinstance(name, str) or not name:
|
| 1023 |
+
continue
|
| 1024 |
+
call: dict[str, Any] = {
|
| 1025 |
+
"type": "function",
|
| 1026 |
+
"function": {
|
| 1027 |
+
"name": name,
|
| 1028 |
+
"arguments": normalize_openai_tool_arguments(
|
| 1029 |
+
function.get("arguments", {})
|
| 1030 |
+
),
|
| 1031 |
+
},
|
| 1032 |
+
}
|
| 1033 |
+
if isinstance(raw_call.get("id"), str) and raw_call["id"]:
|
| 1034 |
+
call["id"] = raw_call["id"]
|
| 1035 |
+
calls.append(call)
|
| 1036 |
+
return calls
|
| 1037 |
+
|
| 1038 |
+
|
| 1039 |
+
def normalize_messages(
|
| 1040 |
+
messages: list[dict[str, Any]],
|
| 1041 |
+
extra_system_instruction: str | None = None,
|
| 1042 |
+
) -> list[dict[str, Any]]:
|
| 1043 |
+
"""Normalize multimodal content while preserving native tool history."""
|
| 1044 |
+
normalized: list[dict[str, Any]] = []
|
| 1045 |
+
for message in messages:
|
| 1046 |
+
raw_role = str(message.get("role", "user")).lower()
|
| 1047 |
+
if raw_role in {"system", "developer"}:
|
| 1048 |
+
role = "system"
|
| 1049 |
+
elif raw_role in {"assistant", "tool"}:
|
| 1050 |
+
role = raw_role
|
| 1051 |
+
else:
|
| 1052 |
+
role = "user"
|
| 1053 |
+
|
| 1054 |
+
entry: dict[str, Any] = {
|
| 1055 |
+
"role": role,
|
| 1056 |
+
"content": text_content(message.get("content")),
|
| 1057 |
+
}
|
| 1058 |
+
if role == "assistant":
|
| 1059 |
+
calls = normalized_tool_calls(message.get("tool_calls"))
|
| 1060 |
+
if calls:
|
| 1061 |
+
entry["tool_calls"] = calls
|
| 1062 |
+
if role == "tool" and isinstance(message.get("tool_call_id"), str):
|
| 1063 |
+
entry["tool_call_id"] = message["tool_call_id"]
|
| 1064 |
+
normalized.append(entry)
|
| 1065 |
+
|
| 1066 |
+
if extra_system_instruction:
|
| 1067 |
+
if normalized and normalized[0]["role"] == "system":
|
| 1068 |
+
normalized[0]["content"] = (
|
| 1069 |
+
f"{normalized[0]['content']}\n\n{extra_system_instruction}"
|
| 1070 |
+
).strip()
|
| 1071 |
+
else:
|
| 1072 |
+
normalized.insert(
|
| 1073 |
+
0, {"role": "system", "content": extra_system_instruction}
|
| 1074 |
+
)
|
| 1075 |
+
return normalized
|
requirements_final.txt
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# PyTorch CUDA 13.0 wheels. ZeroGPU log confirmed torch 2.11.0+cu130.
|
| 2 |
+
--extra-index-url https://download.pytorch.org/whl/cu130
|
| 3 |
+
|
| 4 |
+
# Space/API runtime
|
| 5 |
+
fastapi>=0.115,<1
|
| 6 |
+
pydantic>=2.10,<3
|
| 7 |
+
httpx>=0.27,<1
|
| 8 |
+
gradio==6.22.0
|
| 9 |
+
|
| 10 |
+
# Pin the exact ZeroGPU-compatible PyTorch pair.
|
| 11 |
+
torch==2.11.0
|
| 12 |
+
torchvision==0.26.0
|
| 13 |
+
|
| 14 |
+
# GPTQModel 7.3.2 declares accelerate>=1.13.0, torch>=2.8,
|
| 15 |
+
# safetensors>=0.7 and transformers>=5.4.
|
| 16 |
+
accelerate>=1.13.0,<2
|
| 17 |
+
safetensors>=0.7.0
|
| 18 |
+
pillow>=11.3.0
|
| 19 |
+
|
| 20 |
+
# AWQ runtime verified by the Space startup log.
|
| 21 |
+
transformers==5.14.1
|
| 22 |
+
gptqmodel==7.3.2
|
| 23 |
+
|
| 24 |
+
# Dependencies used by helper modules in this repository.
|
| 25 |
+
tiktoken>=0.9
|
| 26 |
+
sentencepiece>=0.2
|