Erinaldorodrigues commited on
Commit
f5c32d1
·
verified ·
1 Parent(s): 587231e

Upload 4 files

Browse files
Files changed (4) hide show
  1. README_final.md +108 -0
  2. app_final.py +567 -0
  3. openai_compat_final.py +1075 -0
  4. requirements_final.txt +26 -0
README_final.md ADDED
@@ -0,0 +1,108 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: Qwen2.5 Coder 32B AWQ OpenAI API
3
+ emoji: 🚀
4
+ colorFrom: green
5
+ colorTo: yellow
6
+ sdk: gradio
7
+ sdk_version: 6.22.0
8
+ python_version: '3.12'
9
+ app_file: app.py
10
+ pinned: false
11
+ ---
12
+
13
+ # Qwen2.5-Coder-32B AWQ — OpenAI-compatible ZeroGPU API
14
+
15
+ This Space serves `Qwen/Qwen2.5-Coder-32B-Instruct-AWQ` through an
16
+ OpenAI-compatible Chat Completions endpoint on Hugging Face ZeroGPU.
17
+
18
+ The runtime is pinned to the versions that were validated during the Space
19
+ startup work:
20
+
21
+ - PyTorch 2.11.0 / CUDA 13.0 wheels
22
+ - torchvision 0.26.0
23
+ - Transformers 5.14.1
24
+ - GPTQModel 7.3.2
25
+ - Gradio 6.22.0
26
+
27
+ The AWQ model is intentionally loaded lazily from inside the `@spaces.GPU`
28
+ function. This is required by this deployment because AWQ/Marlin performs real
29
+ CUDA work while `from_pretrained()` is running. Do not move model loading back
30
+ to module startup without retesting the Space on ZeroGPU.
31
+
32
+ ## API
33
+
34
+ - `GET /health`
35
+ - `GET /v1/models`
36
+ - `POST /v1/chat/completions`
37
+ - `GET /web-search?q=...`
38
+
39
+ Accepted model names are the real model ID plus the compatibility aliases
40
+ `qwen2.5-coder-32b` and `qwen-coder`. Aliases for unrelated Qwen3 or 14B
41
+ weights are deliberately not accepted.
42
+
43
+ Example client configuration:
44
+
45
+ ```text
46
+ OPENAI_BASE_URL=https://erinaldorodrigues-qwen-coder-api.hf.space/v1
47
+ OPENAI_API_BASE=https://erinaldorodrigues-qwen-coder-api.hf.space/v1
48
+ OPENAI_MODEL=qwen2.5-coder-32b
49
+ WEB_SEARCH_PROVIDER=custom
50
+ WEB_SEARCH_API=https://erinaldorodrigues-qwen-coder-api.hf.space/web-search
51
+ WEB_METHOD=GET
52
+ WEB_QUERY_PARAM=q
53
+ ```
54
+
55
+ If a client requires a non-empty `OPENAI_API_KEY`, it may send one, but the
56
+ current `app.py` does not implement application-level Bearer-token validation.
57
+ Add authentication before exposing private quota or sensitive tools.
58
+
59
+ ## Tool calling
60
+
61
+ The backend accepts OpenAI-style `tools`, `tool_choice`, and
62
+ `parallel_tool_calls`. Tool definitions are normalized for Qwen's native chat
63
+ template and textual `<tool_call>...</tool_call>` outputs are translated back
64
+ to OpenAI `message.tool_calls` objects.
65
+
66
+ `tool_choice="required"` is reconciled with conversation state for OpenClaude:
67
+ when the user explicitly identifies a tool, the choice is narrowed to that
68
+ function; once usable tool evidence is already present, repeated `required`
69
+ choices may be downgraded to `none` so the model can synthesize the answer
70
+ instead of looping.
71
+
72
+ When `parallel_tool_calls=true`, the prompt permits multiple independent tool
73
+ calls. Otherwise generation stops after the first complete tool call.
74
+
75
+ The Space generates tool calls; the calling client remains responsible for
76
+ executing client-side tools and returning their results in subsequent `tool`
77
+ messages. `/web-search` is a separate server-side search endpoint.
78
+
79
+ ## Generation and compatibility
80
+
81
+ The default context limit is 16,384 tokens and default maximum output is 2,048
82
+ tokens. Both can be changed with Space environment variables.
83
+
84
+ `temperature=0` is preserved as greedy generation. Responses report prompt and
85
+ completion token counts, and `finish_reason="length"` is returned when the
86
+ configured output budget is exhausted without a completed tool call.
87
+
88
+ `stream=true` returns OpenAI-style SSE framing. The current implementation
89
+ finishes model generation before emitting the content delta, so it is protocol
90
+ streaming rather than token-by-token low-latency streaming. This is intentional
91
+ until a ZeroGPU-safe streamer/thread implementation is validated.
92
+
93
+ ## Health
94
+
95
+ `/health` reports both the configured model and `model_loaded`. Because model
96
+ loading is lazy, a healthy freshly started process can report
97
+ `model_loaded=false` until the first GPU inference initializes the AWQ model.
98
+
99
+ ## Tests
100
+
101
+ Run:
102
+
103
+ ```bash
104
+ python -m unittest discover -p 'test_*.py'
105
+ ```
106
+
107
+ The suite covers generation helpers, OpenAI/OpenClaude tool flow, tool-call
108
+ parsing, web-search fallbacks, and static integration regressions in `app.py`.
app_final.py ADDED
@@ -0,0 +1,567 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Reliable ZeroGPU backend for the local OpenAI-compatible proxy."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import json
6
+ import os
7
+ import time
8
+ import uuid
9
+ import traceback
10
+ from typing import Any
11
+
12
+ os.environ.setdefault("HF_HUB_DISABLE_PROGRESS_BARS", "1")
13
+ os.environ.setdefault("TRANSFORMERS_DISABLE_DEEPGEMM_LINEAR", "1")
14
+
15
+ import gradio as gr
16
+ import spaces
17
+ import torch
18
+ from fastapi import HTTPException
19
+ from fastapi.responses import JSONResponse, StreamingResponse
20
+ from pydantic import BaseModel, ValidationError
21
+ from starlette.concurrency import run_in_threadpool
22
+ from starlette.middleware.base import BaseHTTPMiddleware
23
+ from starlette.requests import Request
24
+ from transformers import (
25
+ AutoModelForCausalLM,
26
+ AutoTokenizer,
27
+ StoppingCriteria,
28
+ StoppingCriteriaList,
29
+ )
30
+ from generation import (
31
+ gpu_duration_seconds,
32
+ head_tail_token_counts,
33
+ merge_eos_token_ids,
34
+ )
35
+ from openai_compat import (
36
+ analyze_tool_flow,
37
+ indexed_tool_calls,
38
+ normalize_tools,
39
+ resolve_tool_choice,
40
+ select_tools,
41
+ tool_choice_instruction,
42
+ tool_protocol_instruction,
43
+ tool_names,
44
+ )
45
+ from openclaude_compat import (
46
+ TOOL_PROTOCOL_MARKER,
47
+ add_system_instruction,
48
+ has_tool_protocol,
49
+ normalize_openclaude_messages,
50
+ )
51
+ from tool_calls import (
52
+ extract_tool_calls,
53
+ has_complete_tool_call,
54
+ )
55
+ from web_search import SearchUnavailable, search_web
56
+
57
+
58
+ # O Titã: Qwen2.5-Coder-32B nativamente quantizado em 4-bits (AWQ)
59
+ MODEL = os.getenv(
60
+ "MODEL",
61
+ os.getenv("MODEL_ID", "Qwen/Qwen2.5-Coder-32B-Instruct-AWQ"),
62
+ )
63
+
64
+ MAX_CONTEXT_TOKENS = int(os.getenv("MAX_CONTEXT_TOKENS", "16384"))
65
+ MAX_NEW_TOKENS = int(os.getenv("MAX_NEW_TOKENS", "2048"))
66
+ MAX_TOOL_CALL_TOKENS = int(os.getenv("MAX_TOOL_CALL_TOKENS", "2048"))
67
+ MAX_TEMPERATURE = float(os.getenv("MAX_TEMPERATURE", "0.2"))
68
+ PRESERVED_PREFIX_TOKENS = int(os.getenv("PRESERVED_PREFIX_TOKENS", "4096"))
69
+
70
+ tokenizer = AutoTokenizer.from_pretrained(MODEL)
71
+
72
+ # O ZeroGPU só anexa uma GPU real dentro de funções decoradas com
73
+ # @spaces.GPU; no escopo do módulo (startup) não existe CUDA de verdade,
74
+ # apenas uma emulação que aceita `.to("cuda")`/`device_map="auto"` como
75
+ # simples posicionamento de tensores. O carregamento deste modelo AWQ,
76
+ # porém, dispara o kernel Marlin (`awq_marlin_repack`) de forma síncrona
77
+ # dentro do próprio from_pretrained — isso é execução real de kernel CUDA,
78
+ # não posicionamento, e por isso não existe backend CPU para ele (era
79
+ # exatamente esse o erro do seu log). Por isso o carregamento precisa ser
80
+ # adiado para dentro de `gerar`, a única função com GPU real anexada.
81
+ model: AutoModelForCausalLM | None = None
82
+
83
+
84
+ def _ensure_model_loaded() -> None:
85
+ """Carrega o modelo uma única vez, já dentro do contexto com GPU real."""
86
+ global model
87
+ if model is not None:
88
+ return
89
+ print(f"Loading {MODEL} on ZeroGPU (NATIVE AWQ)...", flush=True)
90
+ model = AutoModelForCausalLM.from_pretrained(
91
+ MODEL,
92
+ dtype="auto",
93
+ device_map="auto",
94
+ low_cpu_mem_usage=True,
95
+ )
96
+ model.eval()
97
+ print(f"Model ready on {next(model.parameters()).device}", flush=True)
98
+
99
+
100
+ def _bounded_output_tokens(value: float) -> int:
101
+ try:
102
+ requested = int(value)
103
+ except (TypeError, ValueError):
104
+ requested = MAX_NEW_TOKENS
105
+ return max(1, min(requested, MAX_NEW_TOKENS))
106
+
107
+
108
+ # Buffer para cobrir a compilação JIT do kernel Marlin + carregamento dos
109
+ # pesos quando `gerar` cai num worker "frio" (sem o modelo em memória).
110
+ # É uma estimativa (baseada nos ~99s de compilação que aparecem no seu log);
111
+ # meça o cold start real do seu Space e ajuste. Confira também o teto de
112
+ # duração por chamada da sua tier em
113
+ # https://huggingface.co/docs/hub/spaces-zerogpu antes de subir esse valor —
114
+ # se o teto for menor que isso, a chamada falha com "illegal duration".
115
+ COLD_START_BUFFER_SECONDS = 180
116
+
117
+
118
+ def _gpu_duration(
119
+ messages_json: str,
120
+ __: float,
121
+ max_new_tokens: float,
122
+ *tool_arguments: object,
123
+ ) -> int:
124
+ output_tokens = _bounded_output_tokens(max_new_tokens)
125
+ tool_characters = sum(
126
+ len(value) for value in tool_arguments if isinstance(value, str)
127
+ )
128
+ duration = gpu_duration_seconds(
129
+ len(messages_json) + tool_characters,
130
+ output_tokens,
131
+ MAX_CONTEXT_TOKENS,
132
+ )
133
+ return duration + COLD_START_BUFFER_SECONDS
134
+
135
+
136
+ def _tool_protocol_active(messages: list[object]) -> bool:
137
+ return any(
138
+ isinstance(message, dict)
139
+ and isinstance(message.get("content"), str)
140
+ and TOOL_PROTOCOL_MARKER in message["content"]
141
+ for message in messages
142
+ )
143
+
144
+
145
+ def _native_tools(raw_tools: object) -> list[dict[str, Any]]:
146
+ return normalize_tools(raw_tools)
147
+
148
+
149
+ def _render_prompt(
150
+ messages: list[dict[str, Any]],
151
+ tools: list[dict[str, Any]],
152
+ ) -> str:
153
+ """Render the exact prompt used for generation, with a safe tool fallback."""
154
+ template_kwargs: dict[str, Any] = {
155
+ "tokenize": False,
156
+ "add_generation_prompt": True,
157
+ }
158
+ if tools:
159
+ template_kwargs["tools"] = tools
160
+
161
+ try:
162
+ return tokenizer.apply_chat_template(messages, **template_kwargs)
163
+ except Exception as template_error:
164
+ print(
165
+ f"Jinja Template Warning: {template_error}. Applying fallback.",
166
+ flush=True,
167
+ )
168
+ template_kwargs.pop("tools", None)
169
+ return tokenizer.apply_chat_template(messages, **template_kwargs)
170
+
171
+
172
+ def _prompt_token_count(
173
+ messages: list[dict[str, Any]],
174
+ tools: list[dict[str, Any]],
175
+ output_tokens: int,
176
+ ) -> int:
177
+ """Count the prompt tokens that survive the same context bound as generation."""
178
+ prompt = _render_prompt(messages, tools)
179
+ encoded = tokenizer(
180
+ prompt,
181
+ add_special_tokens=False,
182
+ truncation=False,
183
+ )["input_ids"]
184
+ input_budget = max(1, MAX_CONTEXT_TOKENS - output_tokens)
185
+ return min(len(encoded), input_budget)
186
+
187
+
188
+ def _completion_token_count(text: str) -> int:
189
+ """Count visible generated tokens for OpenAI-compatible usage reporting."""
190
+ return len(
191
+ tokenizer(
192
+ text,
193
+ add_special_tokens=False,
194
+ truncation=False,
195
+ )["input_ids"]
196
+ )
197
+
198
+
199
+ class StopAfterToolCall(StoppingCriteria):
200
+ def __init__(self, prompt_length: int) -> None:
201
+ self.prompt_length = prompt_length
202
+
203
+ def __call__(self, input_ids, scores, **_: object):
204
+ completed = []
205
+ for sequence in input_ids:
206
+ generated = sequence[self.prompt_length :]
207
+ text = tokenizer.decode(generated, skip_special_tokens=False)
208
+ completed.append(has_complete_tool_call(text))
209
+ return torch.tensor(completed, dtype=torch.bool, device=input_ids.device)
210
+
211
+
212
+ @spaces.GPU(duration=_gpu_duration)
213
+ def gerar(
214
+ messages_json: str,
215
+ temperature: float,
216
+ max_new_tokens: float,
217
+ tools_json: str = "[]",
218
+ stop_after_first_tool: bool = True,
219
+ ) -> str:
220
+ _ensure_model_loaded()
221
+ messages = json.loads(messages_json)
222
+ if not isinstance(messages, list):
223
+ raise ValueError("messages_json must contain a JSON list")
224
+ try:
225
+ tools = _native_tools(json.loads(tools_json))
226
+ except (TypeError, ValueError, json.JSONDecodeError):
227
+ tools = []
228
+ if not isinstance(tools, list):
229
+ tools = []
230
+
231
+ output_tokens = _bounded_output_tokens(max_new_tokens)
232
+ tool_mode = _tool_protocol_active(messages) or bool(tools)
233
+ prompt = _render_prompt(messages, tools)
234
+
235
+ inputs = tokenizer(
236
+ prompt,
237
+ return_tensors="pt",
238
+ add_special_tokens=False,
239
+ truncation=False,
240
+ )
241
+ input_budget = max(1, MAX_CONTEXT_TOKENS - output_tokens)
242
+ input_length = inputs["input_ids"].shape[1]
243
+
244
+ if input_length > input_budget:
245
+ head_tokens, tail_tokens = head_tail_token_counts(
246
+ input_length,
247
+ input_budget,
248
+ PRESERVED_PREFIX_TOKENS,
249
+ )
250
+ for key, value in inputs.items():
251
+ if (
252
+ isinstance(value, torch.Tensor)
253
+ and value.ndim == 2
254
+ and value.shape[1] == input_length
255
+ ):
256
+ parts = []
257
+ if head_tokens:
258
+ parts.append(value[:, :head_tokens])
259
+ if tail_tokens:
260
+ parts.append(value[:, -tail_tokens:])
261
+ inputs[key] = torch.cat(parts, dim=1)
262
+
263
+ inputs = inputs.to("cuda")
264
+
265
+ print(
266
+ f"Generation started: input_tokens={inputs['input_ids'].shape[1]} "
267
+ f"max_new_tokens={output_tokens} tool_mode={tool_mode}",
268
+ flush=True,
269
+ )
270
+
271
+ eos_token_ids = merge_eos_token_ids(
272
+ model.generation_config.eos_token_id,
273
+ tokenizer.eos_token_id,
274
+ )
275
+
276
+ generation_kwargs = {
277
+ "max_new_tokens": output_tokens,
278
+ "do_sample": float(temperature) > 0,
279
+ "pad_token_id": tokenizer.pad_token_id or tokenizer.eos_token_id,
280
+ }
281
+ if eos_token_ids is not None:
282
+ generation_kwargs["eos_token_id"] = eos_token_ids
283
+
284
+ if generation_kwargs["do_sample"]:
285
+ generation_kwargs["temperature"] = max(0.01, float(temperature))
286
+ generation_kwargs["top_p"] = 0.8
287
+ generation_kwargs["top_k"] = 20
288
+ generation_kwargs["repetition_penalty"] = 1.05
289
+
290
+ if tool_mode and stop_after_first_tool:
291
+ generation_kwargs["stopping_criteria"] = StoppingCriteriaList(
292
+ [StopAfterToolCall(inputs["input_ids"].shape[1])]
293
+ )
294
+
295
+ with torch.inference_mode():
296
+ output = model.generate(**inputs, **generation_kwargs)
297
+
298
+ generated = output[0][inputs["input_ids"].shape[1] :]
299
+ response = tokenizer.decode(generated, skip_special_tokens=True).strip()
300
+ print(f"Generation completed: output_tokens={generated.shape[0]}", flush=True)
301
+ return response
302
+
303
+
304
+ class ChatCompletionRequest(BaseModel):
305
+ model: str = MODEL
306
+ messages: list[dict[str, Any]]
307
+ temperature: float = 0.2
308
+ max_tokens: int | None = None
309
+ max_completion_tokens: int | None = None
310
+ stream: bool = False
311
+ tools: list[dict[str, Any]] | None = None
312
+ tool_choice: Any = None
313
+ parallel_tool_calls: bool | None = None
314
+
315
+
316
+ def _completion_payload(request: ChatCompletionRequest) -> dict[str, Any]:
317
+ if request.model not in {
318
+ MODEL,
319
+ "qwen-coder",
320
+ "qwen2.5-coder-32b",
321
+ }:
322
+ raise HTTPException(status_code=404, detail=f"Model not available: {request.model}")
323
+
324
+ already_adapted = has_tool_protocol(request.messages)
325
+ flow_state = analyze_tool_flow(request.messages, request.tools or [])
326
+ state_controls_choice = request.tool_choice is None or (
327
+ isinstance(request.tool_choice, str)
328
+ and request.tool_choice.casefold() in {"auto", "required"}
329
+ )
330
+ effective_choice = resolve_tool_choice(request.tool_choice, flow_state)
331
+
332
+ try:
333
+ effective_tools, tool_mode = select_tools(
334
+ request.tools or [], effective_choice
335
+ )
336
+ except ValueError as error:
337
+ raise HTTPException(status_code=400, detail=str(error)) from error
338
+
339
+ instructions = [
340
+ instruction
341
+ for instruction in (
342
+ (
343
+ tool_protocol_instruction(
344
+ effective_tools,
345
+ parallel_tool_calls=request.parallel_tool_calls is True,
346
+ )
347
+ if effective_tools and not has_tool_protocol(request.messages)
348
+ else None
349
+ ),
350
+ tool_choice_instruction(tool_mode, effective_tools),
351
+ (
352
+ flow_state.instruction
353
+ if state_controls_choice and not already_adapted
354
+ else None
355
+ ),
356
+ )
357
+ if instruction
358
+ ]
359
+ instruction = "\n\n".join(instructions) if instructions else None
360
+ max_tokens = request.max_completion_tokens or request.max_tokens or MAX_NEW_TOKENS
361
+
362
+ if effective_tools:
363
+ max_tokens = min(max_tokens, MAX_TOOL_CALL_TOKENS)
364
+ temperature = min(max(float(request.temperature), 0.0), MAX_TEMPERATURE)
365
+
366
+ try:
367
+ normalized_messages = (
368
+ [dict(message) for message in request.messages]
369
+ if already_adapted
370
+ else normalize_openclaude_messages(request.messages)
371
+ )
372
+ prompt_messages = add_system_instruction(
373
+ normalized_messages,
374
+ instruction,
375
+ )
376
+ except ValueError as error:
377
+ raise HTTPException(status_code=400, detail=str(error)) from error
378
+
379
+ bounded_max_tokens = _bounded_output_tokens(max_tokens)
380
+ prompt_tokens = _prompt_token_count(
381
+ prompt_messages,
382
+ effective_tools,
383
+ bounded_max_tokens,
384
+ )
385
+ text = gerar(
386
+ json.dumps(prompt_messages),
387
+ temperature,
388
+ bounded_max_tokens,
389
+ json.dumps(effective_tools, ensure_ascii=False),
390
+ request.parallel_tool_calls is not True,
391
+ )
392
+ completion_tokens = _completion_token_count(text)
393
+
394
+ if effective_tools:
395
+ tool_calls, content = extract_tool_calls(text, tool_names(effective_tools))
396
+ if request.parallel_tool_calls is False:
397
+ tool_calls = tool_calls[:1]
398
+ else:
399
+ tool_calls, content = [], text
400
+
401
+ message: dict[str, Any] = {"role": "assistant", "content": content or None}
402
+
403
+ finish_reason = "stop"
404
+ if tool_calls:
405
+ message["tool_calls"] = tool_calls
406
+ finish_reason = "tool_calls"
407
+ elif completion_tokens >= bounded_max_tokens:
408
+ finish_reason = "length"
409
+ elif effective_tools and has_complete_tool_call(text):
410
+ finish_reason = "stop"
411
+ elif tool_mode in {"required", "forced"}:
412
+ finish_reason = "stop"
413
+
414
+ return {
415
+ "id": f"chatcmpl-{uuid.uuid4().hex}",
416
+ "object": "chat.completion",
417
+ "created": int(time.time()),
418
+ "model": MODEL,
419
+ "choices": [{"index": 0, "message": message, "finish_reason": finish_reason}],
420
+ "usage": {
421
+ "prompt_tokens": prompt_tokens,
422
+ "completion_tokens": completion_tokens,
423
+ "total_tokens": prompt_tokens + completion_tokens,
424
+ },
425
+ }
426
+
427
+
428
+ def health() -> dict[str, Any]:
429
+ return {
430
+ "status": "ok",
431
+ "model": MODEL,
432
+ "model_loaded": model is not None,
433
+ }
434
+
435
+
436
+ def models() -> dict[str, Any]:
437
+ return {
438
+ "object": "list",
439
+ "data": [
440
+ {
441
+ "id": model_id,
442
+ "object": "model",
443
+ "owned_by": "Erinaldorodrigues",
444
+ "context_length": MAX_CONTEXT_TOKENS,
445
+ "max_input_tokens": MAX_CONTEXT_TOKENS,
446
+ "max_output_tokens": MAX_NEW_TOKENS,
447
+ }
448
+ for model_id in dict.fromkeys(("qwen2.5-coder-32b", MODEL))
449
+ ],
450
+ }
451
+
452
+
453
+ def chat_completions(request: ChatCompletionRequest):
454
+ completion = _completion_payload(request)
455
+ if not request.stream:
456
+ return JSONResponse(content=completion)
457
+
458
+ choice = completion["choices"][0]
459
+ chunk_id = completion["id"]
460
+
461
+ def events():
462
+ first = {
463
+ "id": chunk_id,
464
+ "object": "chat.completion.chunk",
465
+ "created": completion["created"],
466
+ "model": MODEL,
467
+ "choices": [{"index": 0, "delta": {"role": "assistant"}, "finish_reason": None}],
468
+ }
469
+ yield f"data: {json.dumps(first)}\n\n"
470
+
471
+ delta: dict[str, Any] = {}
472
+ if choice["message"].get("content"):
473
+ delta["content"] = choice["message"]["content"]
474
+ if choice["message"].get("tool_calls"):
475
+ delta["tool_calls"] = indexed_tool_calls(
476
+ choice["message"]["tool_calls"]
477
+ )
478
+
479
+ body = {**first, "choices": [{"index": 0, "delta": delta, "finish_reason": None}]}
480
+ yield f"data: {json.dumps(body)}\n\n"
481
+
482
+ final = {**first, "choices": [{"index": 0, "delta": {}, "finish_reason": choice["finish_reason"]}]}
483
+ yield f"data: {json.dumps(final)}\n\n"
484
+ yield "data: [DONE]\n\n"
485
+
486
+ return StreamingResponse(
487
+ events(),
488
+ media_type="text/event-stream",
489
+ headers={
490
+ "Cache-Control": "no-cache",
491
+ "X-Accel-Buffering": "no",
492
+ },
493
+ )
494
+
495
+
496
+ demo = gr.Interface(
497
+ fn=gerar,
498
+ inputs=[
499
+ gr.Textbox(label="Messages JSON"),
500
+ gr.Number(value=0.2, label="Temperature"),
501
+ gr.Number(value=512, label="Max Tokens"),
502
+ gr.Textbox(value="[]", label="Tools JSON"),
503
+ gr.Checkbox(value=True, label="Stop after first complete tool call"),
504
+ ],
505
+ outputs="text",
506
+ title="Qwen2.5-Coder-32B AWQ OpenAI-compatible ZeroGPU Backend",
507
+ )
508
+
509
+ class OpenAIRouteMiddleware(BaseHTTPMiddleware):
510
+ async def dispatch(self, request: Request, call_next):
511
+ path = request.url.path.rstrip("/") or "/"
512
+
513
+ if path == "/health" and request.method == "GET":
514
+ return JSONResponse(health())
515
+
516
+ if path == "/web-search" and request.method == "GET":
517
+ query = request.query_params.get("q", "").strip()
518
+ if not query or len(query) > 500:
519
+ return JSONResponse(status_code=400, content={"error": "invalid query"})
520
+ try:
521
+ return JSONResponse(await run_in_threadpool(search_web, query))
522
+ except SearchUnavailable as error:
523
+ return JSONResponse(
524
+ status_code=503,
525
+ content={"error": {"message": str(error) or "search unavailable"}},
526
+ )
527
+ except Exception as error:
528
+ traceback.print_exc()
529
+ return JSONResponse(
530
+ status_code=500,
531
+ content={"error": {"message": f"search error: {error}"}},
532
+ )
533
+
534
+ if path == "/v1/models" and request.method == "GET":
535
+ return JSONResponse(models())
536
+
537
+ if path == "/v1/chat/completions" and request.method == "POST":
538
+ try:
539
+ raw_request = await request.json()
540
+ parsed_request = ChatCompletionRequest(**raw_request)
541
+ except (json.JSONDecodeError, ValidationError, TypeError) as error:
542
+ return JSONResponse(status_code=400, content={"error": {"message": str(error)}})
543
+ try:
544
+ return chat_completions(parsed_request)
545
+ except HTTPException as error:
546
+ return JSONResponse(status_code=error.status_code, content={"error": {"message": error.detail}})
547
+ except Exception as error:
548
+ traceback.print_exc()
549
+ return JSONResponse(
550
+ status_code=500,
551
+ content={"error": {"message": f"internal Space error: {str(error)}"}}
552
+ )
553
+
554
+ return await call_next(request)
555
+
556
+
557
+ import gradio.routes as _groutes
558
+ _original_create_app = _groutes.App.create_app
559
+
560
+ def _create_app_with_openai_routes(*args, **kwargs):
561
+ created = _original_create_app(*args, **kwargs)
562
+ created.add_middleware(OpenAIRouteMiddleware)
563
+ return created
564
+
565
+ _groutes.App.create_app = staticmethod(_create_app_with_openai_routes)
566
+
567
+ demo.queue(default_concurrency_limit=1, max_size=8).launch(show_error=True, ssr_mode=False)
openai_compat_final.py ADDED
@@ -0,0 +1,1075 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Pure OpenAI compatibility helpers used by the Space endpoint."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import json
6
+ import re
7
+ from collections.abc import Mapping
8
+ from dataclasses import dataclass
9
+ from typing import Any
10
+
11
+ from tool_calls import normalize_openai_tool_arguments
12
+
13
+
14
+ EMPTY_PARAMETERS = {"type": "object", "properties": {}}
15
+ # OpenClaude includes human-facing operational manuals in tool descriptions.
16
+ # They are useful to its native client but can consume most of the Qwen context
17
+ # once the same catalog is rendered again in the model prompt. Keep enough
18
+ # context to select and call a tool while preserving the full JSON-schema shape.
19
+ MAX_TOOL_DESCRIPTION_CHARS = 800
20
+ MAX_SCHEMA_DESCRIPTION_CHARS = 240
21
+
22
+ FAILED_RESULT_RE = re.compile(
23
+ r"(?im)(?:"
24
+ r"<tool_use_error>|"
25
+ r"\bexit\s*(?:code)?\s*[:=]?\s*[1-9]\d*\b|"
26
+ r"\bstatus\s*(?:code)?\s*[:=]?\s*[345]\d\d\b|"
27
+ r"^\s*(?:FAILED|ERROR)(?:\s|:)|"
28
+ r"\b[1-9]\d*\s+(?:failed|errors?)\b|"
29
+ r"\b(?:command not found|no such file|permission denied|timed out)\b|"
30
+ r"\b(?:invalid api key|invalid token|unauthorized|forbidden)\b|"
31
+ r"\b(?:invalid tool parameters|inputvalidationerror)\b|"
32
+ r"\b(?:required parameter|schema)[^\n]*(?:missing|not sent)\b|"
33
+ r'"status"\s*:\s*"(?:error|401|403)"|'
34
+ r'"status"\s*:\s*(?:401|403)\b|'
35
+ r"\bHTTP/\S+\s+(?:3\d\d|4\d\d|5\d\d)\b"
36
+ r")"
37
+ )
38
+ VERIFICATION_COMMAND_RE = re.compile(
39
+ r"(?i)(?:"
40
+ r"\bpytest\b|"
41
+ r"\bpython(?:3)?\s+-m\s+(?:unittest|pytest)\b|"
42
+ r"\bpython(?:3)?\s+[^\n;&|]*test[^\n;&|]*\.py\b|"
43
+ r"\b(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?test\b|"
44
+ r"\b(?:cargo|go)\s+test\b|"
45
+ r"\b(?:cargo)\s+check\b|"
46
+ r"\b(?:mvn|gradle)\s+(?:test|check|build)\b|"
47
+ r"\bmake\s+(?:check|test)\b|"
48
+ r"\b(?:npm|pnpm|yarn|bun)\s+(?:run\s+)?(?:build|check|lint)\b|"
49
+ r"(?:^|[\s/])(?:bash\s+)?[^\s;&|]*test[^\s;&|]*\.sh\b|"
50
+ r"\bpython(?:3)?\s+-m\s+py_compile\b|"
51
+ r"\b(?:ruff|mypy|eslint|tsc)\b"
52
+ r")"
53
+ )
54
+ POSITIVE_VERIFICATION_RE = re.compile(
55
+ r"(?im)(?:"
56
+ r"^\s*OK\s*$|"
57
+ r"\bRan\s+\d+\s+tests?\b|"
58
+ r"\b\d+\s+passed\b|"
59
+ r"\bBUILD\s+SUCCESS(?:FUL)?\b|"
60
+ r"\b(?:tests?|checks?)\s+(?:passed|successful)\b|"
61
+ r"\b[A-Z][A-Z0-9_]+_OK\b|"
62
+ r"\(?(?:Bash )?completed (?:successfully )?"
63
+ r"(?:with no|without)(?: textual)? output\)?"
64
+ r")"
65
+ )
66
+ INSPECTION_COMMAND_RE = re.compile(
67
+ r"(?i)^\s*(?:"
68
+ r"cd\b[^;&|]*(?:&&|;)\s*)?"
69
+ r"(?:ls|pwd|find|rg|grep|cat|sed|head|tail|wc|stat|tree|git|cd)"
70
+ r"\b"
71
+ )
72
+ WEB_REQUEST_RE = re.compile(
73
+ r"(?i)\b(?:"
74
+ r"pesquis(?:e|ar|a)|busque|procure|not[ií]cias?|[uú]ltimas?|"
75
+ r"hoje|agora|atual(?:izado|izada|mente)?|search|latest|news|browser|web"
76
+ r")\b"
77
+ )
78
+ WEB_SUBJECT_RE = re.compile(
79
+ r"(?i)\b(?:"
80
+ r"web|internet|pesquis\w*|busc\w*|procur\w*|not[ií]cias?|"
81
+ r"search|latest|news|info|site|p[aá]gina"
82
+ r")\b"
83
+ )
84
+ LOCAL_INSPECTION_RE = re.compile(
85
+ r"(?i)\b(?:"
86
+ r"mem[oó]ria|ram|cpu|processador|disco|armazenamento|hardware|"
87
+ r"sistema|kernel|processos?|servi[cç]os?|rede|endere[cç]o\s+ip|"
88
+ r"gpu|temperatura|bateria|swap|arquivos?|diret[oó]rios?|pastas?"
89
+ r")\b"
90
+ )
91
+ INSPECTION_INTENT_RE = re.compile(
92
+ r"(?i)\b(?:"
93
+ r"verifi(?:que|car|ca[cç][aã]o)|confira|cheque|inspecione|"
94
+ r"mostre|liste|diagnostique|analise|check|inspect|show|list|explore"
95
+ r")\b"
96
+ )
97
+ READ_REQUEST_RE = re.compile(
98
+ r"(?i)\b(?:leia|ler|read|veja|ver|open|abra)\b"
99
+ )
100
+ EXPLICIT_TOOL_REQUEST_RE = re.compile(
101
+ r"(?i)\b(?:use|usar|utilize|utilizar|chame|chamar|call|invoke|"
102
+ r"execute|executar)\s+"
103
+ r"(?:(?:obrigatoriamente|necessariamente|somente|only|just|"
104
+ r"a|o|as|os|the|ferramenta|tool)\s+)*"
105
+ r"(?P<tool>bash|read|write|edit|glob|grep|websearch|webfetch|"
106
+ r"task|agent|notebookedit|lsp)\b"
107
+ )
108
+ IMPLEMENTATION_REQUEST_RE = re.compile(
109
+ r"(?i)\b(?:"
110
+ r"implemente|implement|corrija|corrigir|fix|edite|editar|modify|"
111
+ r"altere|alterar|crie|criar|create|write|escreva|instale|install|"
112
+ r"baixe|download|execute|rode|run|teste|testar|automatiz\w*"
113
+ r")\b"
114
+ )
115
+ PROGRAMMING_CONTEXT_RE = re.compile(
116
+ r"(?i)\b(?:"
117
+ r"arquivo|file|c[oó]digo|code|projeto|project|reposit[oó]rio|repo|"
118
+ r"script|programa|aplica[cç][aã]o|app|fun[cç][aã]o|function|classe|"
119
+ r"m[oó]dulo|module|teste|test|bug|erro|error|build|site|endpoint|"
120
+ r"proxy|api|depend[eê]ncia|package|solu[cç][aã]o|funcionalidade|feature"
121
+ r")\b"
122
+ )
123
+ ACTION_NOW_RE = re.compile(
124
+ r"(?i)\b(?:fa[cç]a|execute|rode|run|do)\s+(?:isso\s+)?agora\b|"
125
+ r"\bdo\s+it\s+now\b"
126
+ )
127
+ NO_TOOLS_RE = re.compile(
128
+ r"(?i)\b(?:"
129
+ r"n[aã]o\s+(?:use|usar|chame|chamar)|"
130
+ r"sem|"
131
+ r"do\s+not\s+(?:use|call)|"
132
+ r"never\s+(?:use|call)|"
133
+ r"without"
134
+ r")\s+(?:as?\s+)?(?:ferramentas?|tools?)\b"
135
+ )
136
+ SIMPLE_GREETING_RE = re.compile(
137
+ r"(?i)^\s*(?:oi|ol[aá]|hello|hi|hey|bom\s+dia|boa\s+tarde|boa\s+noite)"
138
+ r"[\s!,.?]*$"
139
+ )
140
+ OPENCLAUDE_METADATA_BLOCK_RE = re.compile(
141
+ r"<(?P<tag>available-deferred-tools|system-reminder)\b[^>]*>.*?</(?P=tag)>",
142
+ re.DOTALL | re.IGNORECASE,
143
+ )
144
+
145
+
146
+ @dataclass(frozen=True)
147
+ class ToolFlowState:
148
+ """Request-local progress state; no conversation state is stored globally."""
149
+
150
+ active: bool = False
151
+ requires_tool: bool = False
152
+ can_finalize: bool = False
153
+ reason: str = ""
154
+ instruction: str | None = None
155
+ forced_tool: str | None = None
156
+
157
+
158
+ @dataclass(frozen=True)
159
+ class _ToolResultEvent:
160
+ name: str
161
+ arguments: dict[str, Any]
162
+ content: str
163
+ is_error: bool
164
+ batch: int
165
+
166
+
167
+ def _bounded_description(value: Any, limit: int) -> str:
168
+ """Return a compact single-line description suitable for a model prompt."""
169
+ text = re.sub(r"\s+", " ", str(value or "")).strip()
170
+ if len(text) <= limit:
171
+ return text
172
+ shortened = text[: max(1, limit - 1)].rsplit(" ", 1)[0].rstrip()
173
+ return (shortened or text[: limit - 1]).rstrip() + "…"
174
+
175
+
176
+ def _compact_schema_descriptions(value: Any) -> Any:
177
+ """Bound schema prose without removing structural validation information."""
178
+ if isinstance(value, Mapping):
179
+ return {
180
+ key: (
181
+ _bounded_description(raw_value, MAX_SCHEMA_DESCRIPTION_CHARS)
182
+ if key == "description"
183
+ else _compact_schema_descriptions(raw_value)
184
+ )
185
+ for key, raw_value in value.items()
186
+ }
187
+ if isinstance(value, list):
188
+ return [_compact_schema_descriptions(item) for item in value]
189
+ return value
190
+
191
+
192
+ def _content_text(content: Any) -> str:
193
+ if isinstance(content, str):
194
+ return content
195
+ if isinstance(content, list):
196
+ parts: list[str] = []
197
+ for block in content:
198
+ if isinstance(block, Mapping):
199
+ text = block.get("text", block.get("content", ""))
200
+ if text:
201
+ parts.append(str(text))
202
+ elif block is not None:
203
+ parts.append(str(block))
204
+ return "\n".join(parts)
205
+ return "" if content is None else str(content)
206
+
207
+
208
+ def _user_request_text(content: Any) -> str:
209
+ """Remove OpenClaude's injected metadata before classifying user intent.
210
+
211
+ OpenClaude places deferred-tool lists, skill descriptions, and snip markers
212
+ inside a user-role message. Those blocks can contain words such as
213
+ ``create``, ``code``, or ``test``; treating them as the user's request can
214
+ incorrectly force ``tool_choice=required`` for a plain greeting.
215
+ """
216
+ text = _content_text(content)
217
+ previous = None
218
+ while text != previous:
219
+ previous = text
220
+ text = OPENCLAUDE_METADATA_BLOCK_RE.sub("", text)
221
+ return text.strip()
222
+
223
+
224
+ def _call_arguments(value: Any) -> dict[str, Any]:
225
+ if isinstance(value, Mapping):
226
+ return dict(value)
227
+ if isinstance(value, str):
228
+ try:
229
+ parsed = json.loads(value)
230
+ except json.JSONDecodeError:
231
+ return {}
232
+ return dict(parsed) if isinstance(parsed, Mapping) else {}
233
+ return {}
234
+
235
+
236
+ def _is_synthetic_continuation(message: Mapping[str, Any]) -> bool:
237
+ content = message.get("content")
238
+ if isinstance(content, list) and any(
239
+ isinstance(block, Mapping) and block.get("type") == "tool_result"
240
+ for block in content
241
+ ):
242
+ return True
243
+ text = _content_text(content).casefold()
244
+ return (
245
+ not text.strip()
246
+ or "[tool results received]" in text
247
+ or (
248
+ "continue with the task" in text
249
+ and "resume your thought" in text
250
+ )
251
+ or (
252
+ "<system-reminder" in text
253
+ and not re.sub(
254
+ r"<system-reminder\b[^>]*>.*?</system-reminder>",
255
+ "",
256
+ text,
257
+ flags=re.DOTALL | re.IGNORECASE,
258
+ ).strip()
259
+ )
260
+ )
261
+
262
+
263
+ def _current_turn_messages(messages: object) -> list[object]:
264
+ if not isinstance(messages, list):
265
+ return []
266
+ start = 0
267
+ for index, message in enumerate(messages):
268
+ if (
269
+ isinstance(message, Mapping)
270
+ and str(message.get("role", "")).casefold() == "user"
271
+ and not _is_synthetic_continuation(message)
272
+ ):
273
+ start = index
274
+ return messages[start:]
275
+
276
+
277
+ def _tool_result_events(messages: object) -> list[_ToolResultEvent]:
278
+ current_messages = _current_turn_messages(messages)
279
+ calls_by_id: dict[str, tuple[str, dict[str, Any], int]] = {}
280
+ pending_order: list[str] = []
281
+ events: list[_ToolResultEvent] = []
282
+ batch = 0
283
+
284
+ for message in current_messages:
285
+ if not isinstance(message, Mapping):
286
+ continue
287
+ role = str(message.get("role", "")).casefold()
288
+ if role == "assistant":
289
+ raw_calls = message.get("tool_calls") or []
290
+ if raw_calls:
291
+ batch += 1
292
+ for index, raw_call in enumerate(raw_calls):
293
+ if not isinstance(raw_call, Mapping):
294
+ continue
295
+ function = raw_call.get("function")
296
+ if not isinstance(function, Mapping):
297
+ continue
298
+ name = function.get("name")
299
+ if not isinstance(name, str) or not name:
300
+ continue
301
+ call_id = raw_call.get("id")
302
+ if not isinstance(call_id, str) or not call_id:
303
+ call_id = f"__ordered_{len(calls_by_id)}_{index}"
304
+ calls_by_id[call_id] = (
305
+ name,
306
+ _call_arguments(function.get("arguments", {})),
307
+ batch,
308
+ )
309
+ pending_order.append(call_id)
310
+ continue
311
+ if role != "tool":
312
+ continue
313
+
314
+ call_id = message.get("tool_call_id")
315
+ call: tuple[str, dict[str, Any], int] | None = None
316
+ if isinstance(call_id, str) and call_id:
317
+ call = calls_by_id.pop(call_id, None)
318
+ if call_id in pending_order:
319
+ pending_order.remove(call_id)
320
+ elif pending_order:
321
+ fallback_id = pending_order.pop(0)
322
+ call = calls_by_id.pop(fallback_id, None)
323
+
324
+ if call is None:
325
+ explicit_name = message.get("name")
326
+ if not isinstance(explicit_name, str) or not explicit_name:
327
+ continue
328
+ call = (explicit_name, {}, batch)
329
+
330
+ content = _content_text(message.get("content"))
331
+ structured_error = message.get("is_error") is True
332
+ if isinstance(message.get("content"), list):
333
+ structured_error = structured_error or any(
334
+ isinstance(block, Mapping) and block.get("is_error") is True
335
+ for block in message["content"]
336
+ )
337
+ events.append(
338
+ _ToolResultEvent(
339
+ name=call[0],
340
+ arguments=call[1],
341
+ content=content,
342
+ is_error=structured_error or bool(FAILED_RESULT_RE.search(content)),
343
+ batch=call[2],
344
+ )
345
+ )
346
+ return events
347
+
348
+
349
+ def _bash_command(event: _ToolResultEvent) -> str:
350
+ command = event.arguments.get("command", event.arguments.get("cmd", ""))
351
+ return command if isinstance(command, str) else str(command)
352
+
353
+
354
+ def _bash_proves_completion(event: _ToolResultEvent) -> bool:
355
+ if event.is_error:
356
+ return False
357
+ command = _bash_command(event)
358
+ if not VERIFICATION_COMMAND_RE.search(command):
359
+ return False
360
+ return bool(POSITIVE_VERIFICATION_RE.search(event.content))
361
+
362
+
363
+ def _latest_user_request(messages: object) -> str:
364
+ requests: list[str] = []
365
+ if not isinstance(messages, list):
366
+ return ""
367
+ for message in messages:
368
+ if (
369
+ isinstance(message, Mapping)
370
+ and str(message.get("role", "")).casefold() == "user"
371
+ and not _is_synthetic_continuation(message)
372
+ ):
373
+ text = _user_request_text(message.get("content"))
374
+ if text:
375
+ requests.append(text)
376
+ if not requests:
377
+ return ""
378
+ latest = requests[-1]
379
+ if len(requests) > 1 and ACTION_NOW_RE.search(latest):
380
+ return requests[-2] + "\n" + latest
381
+ return latest
382
+
383
+
384
+ def is_simple_greeting(messages: object) -> bool:
385
+ """Identify a greeting that does not need a model or tool prompt.
386
+
387
+ OpenClaude sends its complete tool catalog even for ``ola``. Calling a
388
+ model on ZeroGPU for that turn adds unnecessary queue time, so the API can
389
+ answer it deterministically before inference.
390
+ """
391
+ return bool(SIMPLE_GREETING_RE.fullmatch(_latest_user_request(messages)))
392
+
393
+
394
+ def _explicitly_disables_tools(messages: object) -> bool:
395
+ if not isinstance(messages, list):
396
+ return False
397
+ return any(
398
+ isinstance(message, Mapping)
399
+ and str(message.get("role", "")).casefold()
400
+ in {"system", "developer", "user"}
401
+ and bool(NO_TOOLS_RE.search(_content_text(message.get("content"))))
402
+ for message in messages
403
+ )
404
+
405
+
406
+ def _initial_tool_flow(
407
+ messages: object,
408
+ available_by_fold: Mapping[str, str],
409
+ ) -> ToolFlowState:
410
+ """Force action for concrete first-turn requests instead of accepting plans."""
411
+ request = _latest_user_request(messages)
412
+ if not request or not available_by_fold:
413
+ return ToolFlowState()
414
+
415
+ explicit_tool = EXPLICIT_TOOL_REQUEST_RE.search(request)
416
+ if explicit_tool:
417
+ requested_name = explicit_tool.group("tool").casefold()
418
+ forced_tool = available_by_fold.get(requested_name)
419
+ if forced_tool is None:
420
+ forced_tool = available_by_fold.get(
421
+ {"agent": "task", "task": "agent"}.get(requested_name, "")
422
+ )
423
+ if forced_tool is not None:
424
+ return ToolFlowState(
425
+ active=True,
426
+ requires_tool=True,
427
+ reason=f"the user explicitly requested the {forced_tool} tool",
428
+ instruction=(
429
+ f"OPENCLAUDE FLOW STATE: call {forced_tool} now because the "
430
+ "user explicitly requested it. Do not print a sample call "
431
+ "as prose and do not answer with a plan."
432
+ ),
433
+ forced_tool=forced_tool,
434
+ )
435
+
436
+ if (
437
+ "websearch" in available_by_fold
438
+ and WEB_REQUEST_RE.search(request)
439
+ and WEB_SUBJECT_RE.search(request)
440
+ ):
441
+ return ToolFlowState(
442
+ active=True,
443
+ requires_tool=True,
444
+ reason="the user requested current web research",
445
+ instruction=(
446
+ "OPENCLAUDE FLOW STATE: perform the requested research now. "
447
+ "Call WebSearch with a concise query; do not merely describe how "
448
+ "you would search and do not substitute curl or invented APIs."
449
+ ),
450
+ forced_tool=available_by_fold["websearch"],
451
+ )
452
+
453
+ if (
454
+ "bash" in available_by_fold
455
+ and LOCAL_INSPECTION_RE.search(request)
456
+ and INSPECTION_INTENT_RE.search(request)
457
+ ):
458
+ return ToolFlowState(
459
+ active=True,
460
+ requires_tool=True,
461
+ reason="the user requested inspection of the local system",
462
+ instruction=(
463
+ "OPENCLAUDE FLOW STATE: inspect the local system now. Call Bash "
464
+ "with a safe read-only command that directly answers the request; "
465
+ "do not print a command as prose and do not ask for confirmation."
466
+ ),
467
+ forced_tool=available_by_fold["bash"],
468
+ )
469
+
470
+ if "read" in available_by_fold and READ_REQUEST_RE.search(request):
471
+ return ToolFlowState(
472
+ active=True,
473
+ requires_tool=True,
474
+ reason="the user explicitly requested reading a file",
475
+ instruction=(
476
+ "OPENCLAUDE FLOW STATE: call Read now for the relevant file. "
477
+ "Do not describe a future read operation."
478
+ ),
479
+ forced_tool=available_by_fold["read"],
480
+ )
481
+
482
+ concrete_implementation = bool(
483
+ IMPLEMENTATION_REQUEST_RE.search(request)
484
+ and (
485
+ PROGRAMMING_CONTEXT_RE.search(request)
486
+ or re.search(r"(?i)\bautomatiz\w*\b", request)
487
+ )
488
+ )
489
+ if ACTION_NOW_RE.search(request) or concrete_implementation:
490
+ return ToolFlowState(
491
+ active=True,
492
+ requires_tool=True,
493
+ reason="the user requested immediate tool-backed action",
494
+ instruction=(
495
+ "OPENCLAUDE FLOW STATE: act on the request now by calling one "
496
+ "appropriate available tool. Do not answer with a plan, example "
497
+ "commands, or a request for the user to repeat the task."
498
+ ),
499
+ )
500
+
501
+ # OpenClaude may send ``tool_choice=required`` even for greetings and
502
+ # other conversational turns. Those turns must be allowed to finalize;
503
+ # requiring a synthetic tool call makes a harmless "oi" become a 502.
504
+ return ToolFlowState(reason="no concrete tool action was requested")
505
+
506
+
507
+ def analyze_tool_flow(
508
+ messages: object,
509
+ raw_tools: object,
510
+ ) -> ToolFlowState:
511
+ """Derive whether an agent must continue or may emit its final response."""
512
+ if _explicitly_disables_tools(messages):
513
+ return ToolFlowState(
514
+ can_finalize=True,
515
+ reason="the request explicitly disables all tools",
516
+ )
517
+ available_by_fold = {
518
+ tool["function"]["name"].casefold(): tool["function"]["name"]
519
+ for tool in normalize_tools(raw_tools)
520
+ }
521
+ available = set(available_by_fold)
522
+ events = _tool_result_events(messages)
523
+ if not events:
524
+ return _initial_tool_flow(messages, available_by_fold)
525
+
526
+ # A successful search/fetch is terminal evidence for a research request.
527
+ # This intentionally prevents WebSearch -> WebFetch -> repeated curl loops.
528
+ web_evidence = any(
529
+ event.name.casefold() in {"websearch", "webfetch"}
530
+ and not event.is_error
531
+ and bool(event.content.strip())
532
+ for event in events
533
+ )
534
+
535
+ request = _latest_user_request(messages)
536
+ agentic_intent = not request or bool(
537
+ IMPLEMENTATION_REQUEST_RE.search(request)
538
+ and (
539
+ PROGRAMMING_CONTEXT_RE.search(request)
540
+ or re.search(r"(?i)\bautomatiz\w*\b", request)
541
+ )
542
+ )
543
+ agentic = (
544
+ agentic_intent
545
+ and "bash" in available
546
+ and bool({"edit", "write"} & available)
547
+ )
548
+ dirty = False
549
+ dirty_batch = -1
550
+ agentic_started = False
551
+ last_reason = ""
552
+
553
+ if agentic:
554
+ for event in events:
555
+ name = event.name.casefold()
556
+ if name == "read":
557
+ agentic_started = True
558
+ dirty = True
559
+ dirty_batch = max(dirty_batch, event.batch)
560
+ last_reason = "files were inspected but implementation is still pending"
561
+ elif name in {"edit", "write"}:
562
+ agentic_started = True
563
+ dirty = True
564
+ dirty_batch = max(dirty_batch, event.batch)
565
+ last_reason = "files changed and must be verified with Bash"
566
+ elif event.is_error and agentic_started:
567
+ dirty = True
568
+ dirty_batch = max(dirty_batch, event.batch)
569
+ last_reason = f"{event.name} returned an error that must be recovered"
570
+ elif name == "bash":
571
+ command = _bash_command(event)
572
+ if event.is_error:
573
+ agentic_started = True
574
+ dirty = True
575
+ dirty_batch = max(dirty_batch, event.batch)
576
+ last_reason = "the Bash command or test failed"
577
+ elif INSPECTION_COMMAND_RE.search(command):
578
+ agentic_started = True
579
+ dirty = True
580
+ dirty_batch = max(dirty_batch, event.batch)
581
+ last_reason = "inspection output is not completion evidence"
582
+ elif (
583
+ agentic_started
584
+ and dirty
585
+ and event.batch > dirty_batch
586
+ and _bash_proves_completion(event)
587
+ ):
588
+ dirty = False
589
+ last_reason = "a Bash verification passed after the latest change"
590
+ elif agentic_started and dirty:
591
+ last_reason = "Bash did not provide positive test evidence"
592
+
593
+ if agentic_started and dirty:
594
+ return ToolFlowState(
595
+ active=True,
596
+ requires_tool=True,
597
+ reason=last_reason,
598
+ instruction=(
599
+ "OPENCLAUDE FLOW STATE: the task is not complete. "
600
+ f"Reason: {last_reason}. Call exactly one appropriate tool now; "
601
+ "do not describe a future plan. After reading, edit or write the "
602
+ "implementation. After changes, use Bash to run the requested "
603
+ "tests and continue fixing failures until the output proves success."
604
+ ),
605
+ )
606
+
607
+ if agentic_started and not dirty:
608
+ return ToolFlowState(
609
+ active=True,
610
+ can_finalize=True,
611
+ reason=last_reason,
612
+ instruction=(
613
+ "OPENCLAUDE FLOW STATE: verification passed after the latest "
614
+ "change. Do not call another tool. Report the completed work and "
615
+ "the test evidence directly in Brazilian Portuguese."
616
+ ),
617
+ )
618
+
619
+ if web_evidence:
620
+ return ToolFlowState(
621
+ active=True,
622
+ can_finalize=True,
623
+ reason="usable web evidence is available",
624
+ instruction=(
625
+ "OPENCLAUDE FLOW STATE: usable WebSearch/WebFetch results are "
626
+ "already available. Do not call WebFetch, Bash, curl, or another "
627
+ "tool. Synthesize a concrete answer now from the supplied results, "
628
+ "include useful source links, and never invent API keys or facts."
629
+ ),
630
+ )
631
+
632
+ last_webfetch_error = max(
633
+ (
634
+ index
635
+ for index, event in enumerate(events)
636
+ if event.name.casefold() == "webfetch" and event.is_error
637
+ ),
638
+ default=-1,
639
+ )
640
+ last_websearch_error = max(
641
+ (
642
+ index
643
+ for index, event in enumerate(events)
644
+ if event.name.casefold() == "websearch" and event.is_error
645
+ ),
646
+ default=-1,
647
+ )
648
+ toolsearch_recovered = (
649
+ last_webfetch_error >= 0
650
+ and any(
651
+ index > last_webfetch_error
652
+ and event.name.casefold() == "toolsearch"
653
+ and not event.is_error
654
+ for index, event in enumerate(events)
655
+ )
656
+ )
657
+
658
+ forced_tool: str | None = None
659
+ recovery = ""
660
+ web_error_name = ""
661
+ if last_webfetch_error >= 0:
662
+ web_error_name = "WebFetch"
663
+ if toolsearch_recovered and "webfetch" in available:
664
+ forced_tool = available_by_fold["webfetch"]
665
+ recovery = (
666
+ "Retry WebFetch now with both required fields: url and prompt."
667
+ )
668
+ elif "webfetch" not in available and "toolsearch" in available:
669
+ forced_tool = available_by_fold["toolsearch"]
670
+ recovery = (
671
+ "Load WebFetch by calling ToolSearch with query select:WebFetch."
672
+ )
673
+ elif "webfetch" in available:
674
+ forced_tool = available_by_fold["webfetch"]
675
+ recovery = (
676
+ "Retry WebFetch with both required fields: url and prompt."
677
+ )
678
+ elif "websearch" in available:
679
+ forced_tool = available_by_fold["websearch"]
680
+ recovery = "Recover with WebSearch using a concise, relevant query."
681
+ elif last_websearch_error >= 0 and "websearch" in available:
682
+ web_error_name = "WebSearch"
683
+ forced_tool = available_by_fold["websearch"]
684
+ recovery = "Retry WebSearch using a concise, relevant query."
685
+
686
+ if forced_tool:
687
+ return ToolFlowState(
688
+ active=True,
689
+ requires_tool=True,
690
+ reason=f"{web_error_name} returned an error",
691
+ instruction=(
692
+ f"OPENCLAUDE FLOW STATE: {web_error_name} failed. "
693
+ f"{recovery} Do not answer with a plan and do not invent "
694
+ "credentials, endpoints, or placeholder tokens."
695
+ ),
696
+ forced_tool=forced_tool,
697
+ )
698
+
699
+ # Read already provides the requested evidence. Mark it terminal so
700
+ # OpenClaude's repeated ``tool_choice=required`` does not make a small
701
+ # model call Read forever. Keep generic Bash inspection neutral: the
702
+ # existing flow still lets the model decide how to summarize it.
703
+ last_event = events[-1]
704
+ if (
705
+ last_event.name.casefold() == "read"
706
+ and not last_event.is_error
707
+ and bool(last_event.content.strip())
708
+ ):
709
+ return ToolFlowState(
710
+ active=True,
711
+ can_finalize=True,
712
+ reason="a successful Read result is available",
713
+ instruction=(
714
+ "OPENCLAUDE FLOW STATE: the requested Read tool returned usable "
715
+ "evidence. Do not call another tool; synthesize the answer "
716
+ "directly from the result in Brazilian Portuguese."
717
+ ),
718
+ )
719
+
720
+ return ToolFlowState()
721
+
722
+
723
+ def resolve_tool_choice(
724
+ requested_choice: object,
725
+ state: ToolFlowState,
726
+ ) -> object:
727
+ """Resolve client tool choice against the reconstructed conversation state.
728
+
729
+ A concrete function choice remains authoritative. ``required`` is slightly
730
+ different for OpenClaude: the client repeats it across turns, so the Space
731
+ must narrow it to a detected function, or downgrade it to ``none`` once a
732
+ usable tool result is already available / no concrete tool action exists.
733
+ """
734
+ if isinstance(requested_choice, Mapping):
735
+ return requested_choice
736
+
737
+ requested_mode = (
738
+ requested_choice.casefold()
739
+ if isinstance(requested_choice, str)
740
+ else None
741
+ )
742
+
743
+ if requested_mode == "required":
744
+ if state.requires_tool:
745
+ if state.forced_tool:
746
+ return {
747
+ "type": "function",
748
+ "function": {"name": state.forced_tool},
749
+ }
750
+ return "required"
751
+ if state.can_finalize or state.reason == "no concrete tool action was requested":
752
+ return "none"
753
+ return "required"
754
+
755
+ is_auto = requested_choice is None or requested_mode == "auto"
756
+ if not is_auto:
757
+ return requested_choice
758
+
759
+ if state.can_finalize or state.reason == "no concrete tool action was requested":
760
+ return "none"
761
+ if state.requires_tool:
762
+ if state.forced_tool:
763
+ return {
764
+ "type": "function",
765
+ "function": {"name": state.forced_tool},
766
+ }
767
+ return "required"
768
+ return requested_choice
769
+
770
+
771
+ def normalize_tools(raw_tools: object) -> list[dict[str, Any]]:
772
+ """Return valid function definitions for Qwen's native tool template."""
773
+ if not isinstance(raw_tools, list):
774
+ return []
775
+
776
+ normalized: list[dict[str, Any]] = []
777
+ for raw_tool in raw_tools:
778
+ if not isinstance(raw_tool, Mapping):
779
+ continue
780
+ function = raw_tool.get("function")
781
+ candidate = function if isinstance(function, Mapping) else raw_tool
782
+ name = candidate.get("name")
783
+ if not isinstance(name, str) or not name:
784
+ continue
785
+ parameters = candidate.get(
786
+ "parameters", candidate.get("input_schema", EMPTY_PARAMETERS)
787
+ )
788
+ if not isinstance(parameters, Mapping):
789
+ parameters = EMPTY_PARAMETERS
790
+ normalized.append(
791
+ {
792
+ "type": "function",
793
+ "function": {
794
+ "name": name,
795
+ "description": _bounded_description(
796
+ candidate.get("description"), MAX_TOOL_DESCRIPTION_CHARS
797
+ ),
798
+ "parameters": _compact_schema_descriptions(parameters),
799
+ },
800
+ }
801
+ )
802
+ return normalized
803
+
804
+
805
+ def select_tools(
806
+ raw_tools: object,
807
+ tool_choice: object,
808
+ ) -> tuple[list[dict[str, Any]], str]:
809
+ """Apply OpenAI ``tool_choice`` semantics before prompting the model.
810
+
811
+ The returned mode is one of ``auto``, ``none``, ``required``, or
812
+ ``forced``. A forced choice only exposes the selected function to Qwen,
813
+ which is the most reliable way to enforce it with a native tool template.
814
+ """
815
+ tools = normalize_tools(raw_tools)
816
+ if tool_choice is None:
817
+ return tools, "auto"
818
+
819
+ if isinstance(tool_choice, str):
820
+ mode = tool_choice.casefold()
821
+ if mode == "none":
822
+ return [], "none"
823
+ if mode in {"auto", "required"}:
824
+ if mode == "required" and not tools:
825
+ raise ValueError("tool_choice='required' needs at least one tool")
826
+ return tools, mode
827
+ raise ValueError(f"Unsupported tool_choice: {tool_choice}")
828
+
829
+ if not isinstance(tool_choice, Mapping):
830
+ raise ValueError("tool_choice must be 'auto', 'none', 'required', or a function")
831
+ function = tool_choice.get("function")
832
+ name = function.get("name") if isinstance(function, Mapping) else None
833
+ if tool_choice.get("type") != "function" or not isinstance(name, str) or not name:
834
+ raise ValueError("Forced tool_choice must contain function.name")
835
+
836
+ selected = [
837
+ tool
838
+ for tool in tools
839
+ if tool["function"]["name"].casefold() == name.casefold()
840
+ ]
841
+ if not selected:
842
+ raise ValueError(f"Forced tool is not defined in tools: {name}")
843
+ return selected[:1], "forced"
844
+
845
+
846
+ def tool_names(tools: list[dict[str, Any]]) -> set[str]:
847
+ return {tool["function"]["name"] for tool in tools}
848
+
849
+
850
+ def indexed_tool_calls(calls: list[dict[str, Any]]) -> list[dict[str, Any]]:
851
+ """Add the per-call index required in streamed OpenAI deltas."""
852
+ return [{**call, "index": index} for index, call in enumerate(calls)]
853
+
854
+
855
+ def tool_choice_instruction(mode: str, tools: list[dict[str, Any]]) -> str | None:
856
+ """Supply the constraint that Qwen's template cannot express directly."""
857
+ if mode == "required":
858
+ return "You must call one or more of the available tools in this response."
859
+ if mode == "forced":
860
+ return (
861
+ f"You must call the {tools[0]['function']['name']} tool in this response. "
862
+ "Do not answer with plain text."
863
+ )
864
+ return None
865
+
866
+
867
+ def _schema_example(parameters: object) -> dict[str, Any]:
868
+ if not isinstance(parameters, Mapping):
869
+ return {}
870
+ properties = parameters.get("properties")
871
+ if not isinstance(properties, Mapping):
872
+ return {}
873
+ required = parameters.get("required")
874
+ keys = required if isinstance(required, list) and required else list(properties)[:1]
875
+ example: dict[str, Any] = {}
876
+ for key in keys:
877
+ if not isinstance(key, str):
878
+ continue
879
+ raw_schema = properties.get(key)
880
+ schema = raw_schema if isinstance(raw_schema, Mapping) else {}
881
+ value_type = schema.get("type")
882
+ if value_type in {"integer", "number"}:
883
+ value: Any = 1
884
+ elif value_type == "boolean":
885
+ value = True
886
+ elif value_type == "array":
887
+ value = []
888
+ elif value_type == "object":
889
+ value = {}
890
+ elif "path" in key.casefold():
891
+ value = "/absolute/path"
892
+ elif "query" in key.casefold():
893
+ value = "search terms"
894
+ elif key.casefold() == "url":
895
+ value = "https://example.com"
896
+ else:
897
+ value = "value"
898
+ example[key] = value
899
+ return example
900
+
901
+
902
+ def tool_protocol_instruction(
903
+ tools: list[dict[str, Any]],
904
+ parallel_tool_calls: bool = False,
905
+ ) -> str | None:
906
+ """Return the complete notebook-agent contract enforced by the Space."""
907
+ if not tools:
908
+ return None
909
+ lines = [
910
+ "OPENAI TOOL CALL FORMAT — MANDATORY",
911
+ "You are operating on the user's real notebook, not a simulation.",
912
+ "Always communicate with the user in Brazilian Portuguese (pt-BR).",
913
+ "Perform requested implementation, diagnosis, download, execution, "
914
+ "testing, local inspection, or current web research with the available "
915
+ "tools instead of describing commands or a future plan.",
916
+ "Never claim that a file changed, a command ran, or a test passed unless "
917
+ "a tool result in this conversation proves it.",
918
+ "After WebSearch or WebFetch returns usable evidence, synthesize the "
919
+ "answer from it. Do not fall back to repeated curl calls.",
920
+ "Never invent API keys, tokens, endpoints, or placeholder credentials.",
921
+ "For greetings, small talk, or a self-contained factual answer, respond "
922
+ "directly without a tool unless the flow state below requires one.",
923
+ (
924
+ "When calling tools, emit one or more complete tool calls and no prose, "
925
+ "Markdown, or code fence. Multiple calls are allowed only when they are "
926
+ "independent and can run in parallel."
927
+ if parallel_tool_calls
928
+ else "When calling a tool, emit exactly one call and no prose, Markdown, "
929
+ "or code fence."
930
+ ),
931
+ 'Exact syntax: <tool_call>{"name":"TOOL_NAME","arguments":{"key":"value"}}</tool_call>',
932
+ "Arguments must be valid JSON matching the selected schema.",
933
+ "Available tools:",
934
+ ]
935
+ available_names = {
936
+ str(tool.get("function", {}).get("name", "")).casefold()
937
+ for tool in tools
938
+ if isinstance(tool.get("function"), Mapping)
939
+ }
940
+ if "webfetch" in available_names:
941
+ lines.insert(
942
+ 5,
943
+ "Call only a tool listed below. Follow every tool schema exactly. "
944
+ "WebFetch requires both url and prompt; never omit required fields.",
945
+ )
946
+ else:
947
+ lines.insert(
948
+ 5,
949
+ "Call only a tool listed below. Deferred tools are unavailable in "
950
+ "this backend; explain when a needed capability is not listed "
951
+ "instead of invoking an unlisted tool.",
952
+ )
953
+ first_example: tuple[str, dict[str, Any]] | None = None
954
+ for tool in tools:
955
+ function = tool.get("function")
956
+ if not isinstance(function, Mapping):
957
+ continue
958
+ name = function.get("name")
959
+ if not isinstance(name, str) or not name:
960
+ continue
961
+ parameters = function.get("parameters")
962
+ lines.append(
963
+ json.dumps(
964
+ {
965
+ "name": name,
966
+ "description": str(function.get("description") or ""),
967
+ "parameters": (
968
+ dict(parameters)
969
+ if isinstance(parameters, Mapping)
970
+ else EMPTY_PARAMETERS
971
+ ),
972
+ },
973
+ ensure_ascii=False,
974
+ separators=(",", ":"),
975
+ )
976
+ )
977
+ if first_example is None:
978
+ first_example = (name, _schema_example(parameters))
979
+ if first_example:
980
+ lines.append(
981
+ "Example syntax: <tool_call>"
982
+ + json.dumps(
983
+ {
984
+ "name": first_example[0],
985
+ "arguments": first_example[1],
986
+ },
987
+ ensure_ascii=False,
988
+ separators=(",", ":"),
989
+ )
990
+ + "</tool_call>"
991
+ )
992
+ return "\n".join(lines)
993
+
994
+
995
+ def text_content(content: Any) -> str:
996
+ """Convert text-only OpenAI message blocks into chat-template text."""
997
+ if isinstance(content, str):
998
+ return content
999
+ if isinstance(content, list):
1000
+ return "\n".join(
1001
+ block.get("text", "")
1002
+ for block in content
1003
+ if isinstance(block, Mapping)
1004
+ and block.get("type") in {"text", "input_text"}
1005
+ )
1006
+ return "" if content is None else str(content)
1007
+
1008
+
1009
+ def normalized_tool_calls(raw_calls: object) -> list[dict[str, Any]]:
1010
+ """Keep valid OpenAI calls in the shape Qwen's template understands."""
1011
+ if not isinstance(raw_calls, list):
1012
+ return []
1013
+
1014
+ calls: list[dict[str, Any]] = []
1015
+ for raw_call in raw_calls:
1016
+ if not isinstance(raw_call, Mapping):
1017
+ continue
1018
+ function = raw_call.get("function")
1019
+ if not isinstance(function, Mapping):
1020
+ continue
1021
+ name = function.get("name")
1022
+ if not isinstance(name, str) or not name:
1023
+ continue
1024
+ call: dict[str, Any] = {
1025
+ "type": "function",
1026
+ "function": {
1027
+ "name": name,
1028
+ "arguments": normalize_openai_tool_arguments(
1029
+ function.get("arguments", {})
1030
+ ),
1031
+ },
1032
+ }
1033
+ if isinstance(raw_call.get("id"), str) and raw_call["id"]:
1034
+ call["id"] = raw_call["id"]
1035
+ calls.append(call)
1036
+ return calls
1037
+
1038
+
1039
+ def normalize_messages(
1040
+ messages: list[dict[str, Any]],
1041
+ extra_system_instruction: str | None = None,
1042
+ ) -> list[dict[str, Any]]:
1043
+ """Normalize multimodal content while preserving native tool history."""
1044
+ normalized: list[dict[str, Any]] = []
1045
+ for message in messages:
1046
+ raw_role = str(message.get("role", "user")).lower()
1047
+ if raw_role in {"system", "developer"}:
1048
+ role = "system"
1049
+ elif raw_role in {"assistant", "tool"}:
1050
+ role = raw_role
1051
+ else:
1052
+ role = "user"
1053
+
1054
+ entry: dict[str, Any] = {
1055
+ "role": role,
1056
+ "content": text_content(message.get("content")),
1057
+ }
1058
+ if role == "assistant":
1059
+ calls = normalized_tool_calls(message.get("tool_calls"))
1060
+ if calls:
1061
+ entry["tool_calls"] = calls
1062
+ if role == "tool" and isinstance(message.get("tool_call_id"), str):
1063
+ entry["tool_call_id"] = message["tool_call_id"]
1064
+ normalized.append(entry)
1065
+
1066
+ if extra_system_instruction:
1067
+ if normalized and normalized[0]["role"] == "system":
1068
+ normalized[0]["content"] = (
1069
+ f"{normalized[0]['content']}\n\n{extra_system_instruction}"
1070
+ ).strip()
1071
+ else:
1072
+ normalized.insert(
1073
+ 0, {"role": "system", "content": extra_system_instruction}
1074
+ )
1075
+ return normalized
requirements_final.txt ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # PyTorch CUDA 13.0 wheels. ZeroGPU log confirmed torch 2.11.0+cu130.
2
+ --extra-index-url https://download.pytorch.org/whl/cu130
3
+
4
+ # Space/API runtime
5
+ fastapi>=0.115,<1
6
+ pydantic>=2.10,<3
7
+ httpx>=0.27,<1
8
+ gradio==6.22.0
9
+
10
+ # Pin the exact ZeroGPU-compatible PyTorch pair.
11
+ torch==2.11.0
12
+ torchvision==0.26.0
13
+
14
+ # GPTQModel 7.3.2 declares accelerate>=1.13.0, torch>=2.8,
15
+ # safetensors>=0.7 and transformers>=5.4.
16
+ accelerate>=1.13.0,<2
17
+ safetensors>=0.7.0
18
+ pillow>=11.3.0
19
+
20
+ # AWQ runtime verified by the Space startup log.
21
+ transformers==5.14.1
22
+ gptqmodel==7.3.2
23
+
24
+ # Dependencies used by helper modules in this repository.
25
+ tiktoken>=0.9
26
+ sentencepiece>=0.2