OhBrian commited on
Commit
d6b82a1
·
1 Parent(s): 6b261f5

更新为 LangGraph 工作流模式

Browse files
Files changed (6) hide show
  1. README.md +13 -7
  2. agent.py +466 -139
  3. app.py +16 -9
  4. config.py +4 -0
  5. requirements.txt +1 -0
  6. tools/llm_client.py +283 -18
README.md CHANGED
@@ -14,10 +14,10 @@ hf_oauth_expiration_minutes: 480
14
 
15
  这是 Hugging Face Agents Course Final Assignment 的基础 Space,用于运行并提交 GAIA 风格问题的 Agent 作答结果。
16
 
17
- 当前版本已经改成 LLM 工具调用 Agent:
18
 
19
  - `app.py` 会通过课程评测接口拉取问题并提交答案。
20
- - `agent.py` 中的 `GaiaAgent` 会让 LLM planner 输出 JSON action,决定下一步调用哪个工具。
21
  - `tools/executor.py` 和 `tools/tool_specs.py` 负责注册、描述并执行工具。
22
  - `tools/` 目录包含规则题、附件下载、Python 执行、Excel 计算、结构化网页、网页检索和体育数据工具。
23
  - 登录 Hugging Face 后,界面会使用当前 HF 用户名提交答案。
@@ -68,18 +68,22 @@ Space 配置参考:https://huggingface.co/docs/hub/spaces-config-reference
68
  6. 维护 gold 文件:`gold_local.jsonl` 每行包含 `task_id`、`expected_answer`、`match_type` 和备注。没有标准答案的问题只做流程测试,不纳入正确率统计。
69
  7. 分层运行:先跑 1 到 3 道 smoke test,再跑全部本地题;确认稳定后再通过 Space 按官方流程提交。
70
 
71
- 当前代码采用 LLM JSON action loop:LLM 每一步只能输出一个 JSON 对象,要么调用工具,要么返回最终答案。Python 不再按固定顺序手动选择 solver,而是只负责执行 LLM 指定的工具。
72
 
73
- 工具调用格式:
 
 
 
 
74
 
75
  ```json
76
- {"thought":"需要先查 Wikipedia 表格","action":"wikipedia_tool","args":{"query":"1928 Summer Olympics athletes"}}
77
  ```
78
 
79
  最终答案格式:
80
 
81
  ```json
82
- {"thought":"工具已给出高置信答案","action":"final_answer","answer":"CUB","confidence":"high"}
83
  ```
84
 
85
  当前仍采用 40% 优先策略:为了先冲过最低分,不处理音频和视频题,只集中处理文本、网页、Python、Excel、体育统计和少量确定性规则题。
@@ -102,7 +106,9 @@ Space 配置参考:https://huggingface.co/docs/hub/spaces-config-reference
102
  python3 gaia_local_eval.py
103
  ```
104
 
105
- 注意:现在本地回归会调用 LLM planner,因此需要 `HF_TOKEN`,并会消耗少量 Hugging Face Inference Providers 额度。这个 token 必须具备 `Make calls to Inference Providers` 权限,否则第一步 planner 请求会返回 403。`MAX_AGENT_STEPS` 可用于限制每题最多工具调用步数。
 
 
106
 
107
  如果本地没有安装依赖,先执行:
108
 
 
14
 
15
  这是 Hugging Face Agents Course Final Assignment 的基础 Space,用于运行并提交 GAIA 风格问题的 Agent 作答结果。
16
 
17
+ 当前版本已经改成 LangGraph 类型工作流 Agent:
18
 
19
  - `app.py` 会通过课程评测接口拉取问题并提交答案。
20
+ - `agent.py` 中的 `GaiaAgent` 会先让 LLM 判断问题类型,再由 LangGraph 路由到对应工具节点。
21
  - `tools/executor.py` 和 `tools/tool_specs.py` 负责注册、描述并执行工具。
22
  - `tools/` 目录包含规则题、附件下载、Python 执行、Excel 计算、结构化网页、网页检索和体育数据工具。
23
  - 登录 Hugging Face 后,界面会使用当前 HF 用户名提交答案。
 
68
  6. 维护 gold 文件:`gold_local.jsonl` 每行包含 `task_id`、`expected_answer`、`match_type` 和备注。没有标准答案的问题只做流程测试,不纳入正确率统计。
69
  7. 分层运行:先跑 1 到 3 道 smoke test,再跑全部本地题;确认稳定后再通过 Space 按官方流程提交。
70
 
71
+ 当前代码采用 LangGraph 三段式工作流:
72
 
73
+ 1. `classify`:LLM 只判断问题类型,例如 `direct_text`、`python_code`、`spreadsheet`、`wikipedia`、`sports`、`web_url`、`web_search`、`attachment_text`、`audio_media`、`video_media`、`vision_image`、`unknown`。
74
+ 2. 类型节点:LangGraph 根据问题类型调用对应工具节点,例如 `direct_answer_tool`、`python_tool`、`spreadsheet_tool`、`wikipedia_tool`、`sports_tool`、`web_read_tool`、`web_search_tool`。
75
+ 3. `finalize`:LLM 只把工具结果格式化为题目要求的最终答案;如果 LLM 调用失败,则使用工具候选答案兜底。
76
+
77
+ 分类输出格式:
78
 
79
  ```json
80
+ {"question_type":"wikipedia","confidence":"high","reason":"Wikipedia structured table question","query":"1928 Summer Olympics athletes"}
81
  ```
82
 
83
  最终答案格式:
84
 
85
  ```json
86
+ {"answer":"CUB","confidence":"high"}
87
  ```
88
 
89
  当前仍采用 40% 优先策略:为了先冲过最低分,不处理音频和视频题,只集中处理文本、网页、Python、Excel、体育统计和少量确定性规则题。
 
106
  python3 gaia_local_eval.py
107
  ```
108
 
109
+ 注意:现在本地回归会优先调用 LLM 做题型分类,并在最后调用 LLM 做答案格式化,因此需要 `HF_TOKEN`,并会消耗少量 Hugging Face Inference Providers 额度。这个 token 必须具备 `Make calls to Inference Providers` 权限,否则 LLM 请求会返回 403。分类失败或格式化失败时,工作流会进入兜底逻辑,尽量使用已有工具候选答案。`MAX_AGENT_STEPS` 可用于限制兜底阶段最多尝试的工具数。
110
+
111
+ 默认不启用 `response_format={"type":"json_object"}`,因为 HF Router 的部分 provider 对这个参数支持不稳定,可能返回空 `message.content`。如果你确认当前模型/provider 支持 JSON mode,可以设置 `HF_PLANNER_USE_RESPONSE_FORMAT=1` 再启用。
112
 
113
  如果本地没有安装依赖,先执行:
114
 
agent.py CHANGED
@@ -1,190 +1,517 @@
1
  import json
2
- from typing import Any
 
 
3
 
4
  from config import HF_TEXT_MODEL, HF_VISION_MODEL, MAX_AGENT_STEPS, MAX_TOOL_OUTPUT_CHARS
5
- from tools.common import normalize_answer, truncate_text
6
- from tools.executor import execute_tool, tool_prompt
7
- from tools.llm_client import AgentAction, plan_next_action
 
 
 
 
 
 
 
 
 
8
  from tools.types import SolverResult
9
 
10
 
11
- SYSTEM_PROMPT = """
12
- You are a GAIA tool-using agent. You must decide which tool to call based on the question, file name,
13
- and prior tool observations.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
 
15
- Rules:
16
- 1. You control tool selection. Do not ask the user for clarification.
17
- 2. Use low-cost deterministic tools before generic web search when they fit.
18
- 3. Do not guess. If enabled tools cannot solve a task, return final_answer "无法确定".
19
- 4. Audio, video, and image/chess tools are currently disabled. If a task requires them, return "无法确定".
20
- 5. Every response must be one valid JSON object and nothing else.
21
- 6. To call a tool, output:
22
- {"thought":"...","action":"tool_name","args":{...}}
23
- 7. To finish, output:
24
- {"thought":"...","action":"final_answer","answer":"...","confidence":"high|medium|low"}
25
- 8. Final answers must obey the requested format exactly. Do not include explanations.
26
- 9. Never answer in plain text. Even if the answer is obvious, return JSON.
27
- 10. For self-contained text/table/regex/list questions, call direct_answer_tool first.
28
- """.strip()
 
 
29
 
30
 
31
  class GaiaAgent:
32
- """LLM 主导的 JSON action-loop Agent。"""
33
 
34
  def __init__(self):
35
- print("GAIA LLM 工具调用 Agent 已初始化。")
36
  print(f"文本模型:{HF_TEXT_MODEL}")
37
  print(f"视觉模型:{HF_VISION_MODEL or '未启用'}")
38
- print(f"最大 Agent 步数:{MAX_AGENT_STEPS}")
 
39
 
40
  def answer_task(self, question: str, task_id: str = "", file_name: str = "") -> SolverResult:
41
  print(f"Agent 收到问题(前 100 个字符):{question[:100]}...")
42
- context = {
43
  "question": question,
44
  "task_id": task_id,
45
  "file_name": file_name,
 
46
  }
47
- messages = self._initial_messages(context)
48
- trace: list[dict[str, Any]] = []
49
- best_tool_answer: dict[str, Any] | None = None
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
 
51
- for step_index in range(1, MAX_AGENT_STEPS + 1):
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
  try:
53
- action = plan_next_action(messages)
54
- except Exception as exc:
55
- trace.append(
56
- {
57
- "step": step_index,
58
- "event": "planner_error",
59
- "error": str(exc),
60
- }
61
  )
62
- return SolverResult(
63
- "无法确定",
64
- source="llm_agent.planner_error",
65
- confidence="low",
66
- evidence=self._trace_text(trace),
67
- error=str(exc),
 
 
 
68
  )
 
 
 
69
 
70
- trace.append(
71
- {
72
- "step": step_index,
73
- "event": "planner_action",
74
- "action": action.action,
75
- "args": action.args,
76
- "thought": action.thought,
77
- }
78
- )
 
 
 
 
 
 
 
 
79
 
80
- if action.is_final:
81
- final_answer = self._extract_final_answer(action)
82
- return SolverResult(
83
- final_answer or "无法确定",
84
- source="llm_agent.final",
85
- confidence=action.confidence,
86
- evidence=self._trace_text(trace),
87
- )
88
 
89
- observation = execute_tool(action.action, action.args, context)
90
- if observation.get("answer"):
91
- best_tool_answer = observation
92
- compact_observation = self._compact_observation(observation)
93
- trace.append(
 
 
 
 
 
 
94
  {
95
- "step": step_index,
96
- "event": "tool_observation",
97
  "observation": compact_observation,
98
- }
99
- )
 
100
 
101
- messages.append(
102
- {
103
- "role": "assistant",
104
- "content": json.dumps(
105
- {
106
- "thought": action.thought,
107
- "action": action.action,
108
- "args": action.args,
109
- },
110
- ensure_ascii=False,
111
- ),
112
- }
113
- )
114
- messages.append(
115
- {
116
- "role": "user",
117
- "content": (
118
- "TOOL_OBSERVATION:\n"
119
- + json.dumps(compact_observation, ensure_ascii=False)
120
- + "\n\nDecide the next tool call or return final_answer as JSON."
121
- ),
122
- }
123
- )
124
 
125
- if best_tool_answer and best_tool_answer.get("answer"):
126
- # 达到步数上限时,使用模型已经选择过的工具返回的候选答案兜底。
127
- return SolverResult(
128
- normalize_answer(str(best_tool_answer["answer"])),
129
- source=f"llm_agent.step_limit_best_tool:{best_tool_answer.get('source')}",
130
- confidence=str(best_tool_answer.get("confidence", "medium")),
131
- evidence=self._trace_text(trace),
132
- error="达到 MAX_AGENT_STEPS,使用最佳工具候选答案。",
133
- )
134
 
135
- return SolverResult(
136
- "无法确定",
137
- source="llm_agent.step_limit",
138
- confidence="low",
139
- evidence=self._trace_text(trace),
140
- error="达到 MAX_AGENT_STEPS,且没有工具候选答案。",
141
  )
142
 
143
- def __call__(self, question: str, task_id: str = "", file_name: str = "") -> str:
144
- result = self.answer_task(question, task_id=task_id, file_name=file_name)
145
- print(
146
- f"Agent 返回:answer={result.answer!r}, source={result.source}, "
147
- f"confidence={result.confidence}, error={result.error}"
148
- )
149
- return normalize_answer(result.answer or "无法确定")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
150
 
151
- def _initial_messages(self, context: dict[str, str]) -> list[dict[str, str]]:
152
- task_payload = {
153
- "task_id": context.get("task_id", ""),
154
- "file_name": context.get("file_name", ""),
155
- "question": context["question"],
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
156
  }
157
- user_prompt = (
158
- "Available tools:\n"
159
- + tool_prompt()
160
- + "\n\nTask:\n"
161
- + json.dumps(task_payload, ensure_ascii=False)
162
- + "\n\nChoose the first tool call or return final_answer. Output JSON only.\n"
163
- + "Example tool call: {\"thought\":\"self-contained rule question\",\"action\":\"direct_answer_tool\",\"args\":{}}\n"
164
- + "Example final answer: {\"thought\":\"tool returned the answer\",\"action\":\"final_answer\",\"answer\":\"right\",\"confidence\":\"high\"}"
165
  )
166
- return [
167
- {"role": "system", "content": SYSTEM_PROMPT},
168
- {"role": "user", "content": user_prompt},
169
- ]
170
-
171
- def _extract_final_answer(self, action: AgentAction) -> str:
172
- answer = action.answer
173
- if answer is None:
174
- answer = action.args.get("answer")
175
- if answer is None:
176
- return "无法确定"
177
- return normalize_answer(str(answer))
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
178
 
179
  def _compact_observation(self, observation: dict[str, Any]) -> dict[str, Any]:
180
  compact = dict(observation)
181
  if compact.get("evidence"):
182
  compact["evidence"] = truncate_text(str(compact["evidence"]), MAX_TOOL_OUTPUT_CHARS)
 
 
 
 
183
  return compact
184
 
185
  def _trace_text(self, trace: list[dict[str, Any]]) -> str:
186
  return truncate_text(json.dumps(trace, ensure_ascii=False, indent=2), MAX_TOOL_OUTPUT_CHARS)
187
 
 
 
 
188
 
189
  # 兼容原模板里的 BasicAgent 名称。
190
  BasicAgent = GaiaAgent
 
1
  import json
2
+ from typing import Any, TypedDict
3
+
4
+ from langgraph.graph import END, StateGraph
5
 
6
  from config import HF_TEXT_MODEL, HF_VISION_MODEL, MAX_AGENT_STEPS, MAX_TOOL_OUTPUT_CHARS
7
+ from tools.common import (
8
+ AUDIO_VIDEO_EXTENSIONS,
9
+ IMAGE_EXTENSIONS,
10
+ SPREADSHEET_EXTENSIONS,
11
+ TEXT_EXTENSIONS,
12
+ extract_urls,
13
+ is_youtube_url,
14
+ normalize_answer,
15
+ truncate_text,
16
+ )
17
+ from tools.executor import execute_tool
18
+ from tools.llm_client import classify_question_type, format_final_answer_with_llm
19
  from tools.types import SolverResult
20
 
21
 
22
+ QUESTION_TYPE_SPECS = [
23
+ {
24
+ "type": "direct_text",
25
+ "tool": "direct_answer_tool",
26
+ "description": "自包含纯文本、表格、反向字符串、列表筛选、简单规则或正则式问题。",
27
+ },
28
+ {
29
+ "type": "python_code",
30
+ "tool": "python_tool",
31
+ "description": "带 .py 附件,需要运行或静态分析 Python 脚本得到输出。",
32
+ },
33
+ {
34
+ "type": "spreadsheet",
35
+ "tool": "spreadsheet_tool",
36
+ "description": "带 .xlsx/.xls 附件,需要读取表格并计算。",
37
+ },
38
+ {
39
+ "type": "wikipedia",
40
+ "tool": "wikipedia_tool",
41
+ "description": "Wikipedia、百科条目、人物作品、Featured Article、奥运表格等结构化网页问题。",
42
+ },
43
+ {
44
+ "type": "sports",
45
+ "tool": "sports_tool",
46
+ "description": "体育统计、球队、赛季、球员数据问题。",
47
+ },
48
+ {
49
+ "type": "web_url",
50
+ "tool": "web_read_tool",
51
+ "description": "问题中给出普通网页 URL,需要读取该 URL 内容。",
52
+ },
53
+ {
54
+ "type": "web_search",
55
+ "tool": "web_search_tool",
56
+ "description": "需要开放网页检索,但没有明确可直接解析的专用工具。",
57
+ },
58
+ {
59
+ "type": "attachment_text",
60
+ "tool": "attachment_text_tool",
61
+ "description": "带 .txt/.csv/.json/.md 等纯文本附件,需要读取附件内容。",
62
+ },
63
+ {
64
+ "type": "audio_media",
65
+ "tool": "audio_tool",
66
+ "description": "音频转写题,例如 .mp3/.wav 附件;当前工具禁用。",
67
+ },
68
+ {
69
+ "type": "video_media",
70
+ "tool": "video_tool",
71
+ "description": "视频或 YouTube 分析题;当前工具禁用。",
72
+ },
73
+ {
74
+ "type": "vision_image",
75
+ "tool": "vision_tool",
76
+ "description": "图片、棋盘图或视觉识别题;当前工具禁用。",
77
+ },
78
+ {
79
+ "type": "unknown",
80
+ "tool": "fallback",
81
+ "description": "无法可靠判断类型时进入兜底流程。",
82
+ },
83
+ ]
84
+
85
+ QUESTION_TYPE_TO_NODE = {
86
+ "direct_text": "direct_text",
87
+ "python_code": "python_code",
88
+ "spreadsheet": "spreadsheet",
89
+ "wikipedia": "wikipedia",
90
+ "sports": "sports",
91
+ "web_url": "web_url",
92
+ "web_search": "web_search",
93
+ "attachment_text": "attachment_text",
94
+ "audio_media": "unsupported_media",
95
+ "video_media": "unsupported_media",
96
+ "vision_image": "unsupported_media",
97
+ "unknown": "fallback",
98
+ }
99
+
100
 
101
+ class GaiaWorkflowState(TypedDict, total=False):
102
+ question: str
103
+ task_id: str
104
+ file_name: str
105
+ question_type: str
106
+ type_confidence: str
107
+ type_reason: str
108
+ type_query: str
109
+ observation: dict[str, Any]
110
+ fallback_used: bool
111
+ answer: str
112
+ source: str
113
+ confidence: str
114
+ evidence: str
115
+ error: str
116
+ trace: list[dict[str, Any]]
117
 
118
 
119
  class GaiaAgent:
120
+ """LangGraph 类型路由工作流 Agent。"""
121
 
122
  def __init__(self):
123
+ print("GAIA LangGraph 类型工作流 Agent 已初始化。")
124
  print(f"文本模型:{HF_TEXT_MODEL}")
125
  print(f"视觉模型:{HF_VISION_MODEL or '未启用'}")
126
+ print(f"最大兜底工具数:{MAX_AGENT_STEPS}")
127
+ self.workflow = self._build_workflow()
128
 
129
  def answer_task(self, question: str, task_id: str = "", file_name: str = "") -> SolverResult:
130
  print(f"Agent 收到问题(前 100 个字符):{question[:100]}...")
131
+ initial_state: GaiaWorkflowState = {
132
  "question": question,
133
  "task_id": task_id,
134
  "file_name": file_name,
135
+ "trace": [],
136
  }
137
+ try:
138
+ final_state = self.workflow.invoke(initial_state)
139
+ except Exception as exc:
140
+ return SolverResult(
141
+ "无法确定",
142
+ source="langgraph.exception",
143
+ confidence="low",
144
+ evidence="",
145
+ error=str(exc),
146
+ )
147
+
148
+ return SolverResult(
149
+ final_state.get("answer") or "无法确定",
150
+ source=final_state.get("source", "langgraph.final"),
151
+ confidence=final_state.get("confidence", "low"),
152
+ evidence=final_state.get("evidence", ""),
153
+ error=final_state.get("error", ""),
154
+ )
155
+
156
+ def __call__(self, question: str, task_id: str = "", file_name: str = "") -> str:
157
+ result = self.answer_task(question, task_id=task_id, file_name=file_name)
158
+ print(
159
+ f"Agent 返回:answer={result.answer!r}, source={result.source}, "
160
+ f"confidence={result.confidence}, error={result.error}"
161
+ )
162
+ return normalize_answer(result.answer or "无法确定")
163
+
164
+ def _build_workflow(self):
165
+ graph = StateGraph(GaiaWorkflowState)
166
+ graph.add_node("classify", self._classify_node)
167
+ graph.add_node("direct_text", self._direct_text_node)
168
+ graph.add_node("python_code", self._python_node)
169
+ graph.add_node("spreadsheet", self._spreadsheet_node)
170
+ graph.add_node("wikipedia", self._wikipedia_node)
171
+ graph.add_node("sports", self._sports_node)
172
+ graph.add_node("web_url", self._web_url_node)
173
+ graph.add_node("web_search", self._web_search_node)
174
+ graph.add_node("attachment_text", self._attachment_text_node)
175
+ graph.add_node("unsupported_media", self._unsupported_media_node)
176
+ graph.add_node("fallback", self._fallback_node)
177
+ graph.add_node("finalize", self._finalize_node)
178
+
179
+ graph.set_entry_point("classify")
180
+ graph.add_conditional_edges("classify", self._route_after_classification)
181
+ for node_name in (
182
+ "direct_text",
183
+ "python_code",
184
+ "spreadsheet",
185
+ "wikipedia",
186
+ "sports",
187
+ "web_url",
188
+ "web_search",
189
+ "attachment_text",
190
+ "unsupported_media",
191
+ ):
192
+ graph.add_conditional_edges(
193
+ node_name,
194
+ self._route_after_tool,
195
+ {"fallback": "fallback", "finalize": "finalize"},
196
+ )
197
+ graph.add_edge("fallback", "finalize")
198
+ graph.add_edge("finalize", END)
199
+ return graph.compile()
200
 
201
+ def _classify_node(self, state: GaiaWorkflowState) -> GaiaWorkflowState:
202
+ try:
203
+ classification = classify_question_type(
204
+ question=state["question"],
205
+ task_id=state.get("task_id", ""),
206
+ file_name=state.get("file_name", ""),
207
+ type_specs=QUESTION_TYPE_SPECS,
208
+ )
209
+ question_type = classification.question_type
210
+ error = state.get("error", "")
211
+ trace_event = {
212
+ "event": "classify",
213
+ "question_type": question_type,
214
+ "confidence": classification.confidence,
215
+ "reason": classification.reason,
216
+ "query": classification.query,
217
+ }
218
+ return {
219
+ "question_type": question_type,
220
+ "type_confidence": classification.confidence,
221
+ "type_reason": classification.reason,
222
+ "type_query": classification.query,
223
+ "error": error,
224
+ "trace": self._append_trace(state, trace_event),
225
+ }
226
+ except Exception as exc:
227
+ trace_event = {
228
+ "event": "classify_error",
229
+ "question_type": "unknown",
230
+ "error": str(exc),
231
+ }
232
+ return {
233
+ "question_type": "unknown",
234
+ "type_confidence": "low",
235
+ "type_reason": "LLM 分类失败,进入兜底流程。",
236
+ "type_query": "",
237
+ "error": self._join_error(state.get("error", ""), f"classifier_error={exc}"),
238
+ "trace": self._append_trace(state, trace_event),
239
+ }
240
+
241
+ def _route_after_classification(self, state: GaiaWorkflowState) -> str:
242
+ return QUESTION_TYPE_TO_NODE.get(state.get("question_type", "unknown"), "fallback")
243
+
244
+ def _route_after_tool(self, state: GaiaWorkflowState) -> str:
245
+ if state.get("fallback_used"):
246
+ return "finalize"
247
+ if state.get("question_type") in {"audio_media", "video_media", "vision_image"}:
248
+ return "finalize"
249
+ observation = state.get("observation", {})
250
+ if self._observation_is_useful(observation):
251
+ return "finalize"
252
+ return "fallback"
253
+
254
+ def _direct_text_node(self, state: GaiaWorkflowState) -> GaiaWorkflowState:
255
+ return self._run_tool_node(state, "direct_answer_tool", {})
256
+
257
+ def _python_node(self, state: GaiaWorkflowState) -> GaiaWorkflowState:
258
+ return self._run_tool_node(state, "python_tool", {})
259
+
260
+ def _spreadsheet_node(self, state: GaiaWorkflowState) -> GaiaWorkflowState:
261
+ return self._run_tool_node(state, "spreadsheet_tool", {})
262
+
263
+ def _wikipedia_node(self, state: GaiaWorkflowState) -> GaiaWorkflowState:
264
+ query = state.get("type_query") or state["question"]
265
+ return self._run_tool_node(state, "wikipedia_tool", {"query": query})
266
+
267
+ def _sports_node(self, state: GaiaWorkflowState) -> GaiaWorkflowState:
268
+ return self._run_tool_node(state, "sports_tool", {})
269
+
270
+ def _web_url_node(self, state: GaiaWorkflowState) -> GaiaWorkflowState:
271
+ observation = self._read_urls_observation(state)
272
+ return self._state_with_observation(state, observation, "web_url")
273
+
274
+ def _web_search_node(self, state: GaiaWorkflowState) -> GaiaWorkflowState:
275
+ observation = self._search_and_read_observation(state)
276
+ return self._state_with_observation(state, observation, "web_search")
277
+
278
+ def _attachment_text_node(self, state: GaiaWorkflowState) -> GaiaWorkflowState:
279
+ return self._run_tool_node(state, "attachment_text_tool", {})
280
+
281
+ def _unsupported_media_node(self, state: GaiaWorkflowState) -> GaiaWorkflowState:
282
+ question_type = state.get("question_type")
283
+ tool_name = {
284
+ "audio_media": "audio_tool",
285
+ "video_media": "video_tool",
286
+ "vision_image": "vision_tool",
287
+ }.get(question_type, "vision_tool")
288
+ return self._run_tool_node(state, tool_name, {})
289
+
290
+ def _fallback_node(self, state: GaiaWorkflowState) -> GaiaWorkflowState:
291
+ observation = self._run_fallback_tools(state)
292
+ return self._state_with_observation(
293
+ {**state, "fallback_used": True},
294
+ observation,
295
+ "fallback",
296
+ )
297
+
298
+ def _finalize_node(self, state: GaiaWorkflowState) -> GaiaWorkflowState:
299
+ observation = state.get("observation", {})
300
+ candidate_answer = str(observation.get("answer") or "").strip()
301
+ source = str(observation.get("source") or observation.get("tool") or "no_tool")
302
+ confidence = str(observation.get("confidence") or "low")
303
+ evidence = str(observation.get("evidence") or "")
304
+ final_error = state.get("error", "")
305
+
306
+ if candidate_answer or evidence:
307
  try:
308
+ formatted = format_final_answer_with_llm(
309
+ question=state["question"],
310
+ question_type=state.get("question_type", "unknown"),
311
+ candidate_answer=candidate_answer,
312
+ evidence=truncate_text(evidence, MAX_TOOL_OUTPUT_CHARS),
313
+ source=source,
314
+ confidence=confidence,
 
315
  )
316
+ answer = formatted.answer or "无法确定"
317
+ confidence = formatted.confidence
318
+ final_error = self._join_error(final_error, observation.get("error", ""))
319
+ except Exception as exc:
320
+ answer = normalize_answer(candidate_answer) if candidate_answer else "无法确定"
321
+ final_error = self._join_error(
322
+ final_error,
323
+ observation.get("error", ""),
324
+ f"final_formatter_error={exc}",
325
  )
326
+ else:
327
+ answer = "无法确定"
328
+ final_error = self._join_error(final_error, observation.get("error", "工具没有返回可用证据。"))
329
 
330
+ trace = self._append_trace(
331
+ state,
332
+ {
333
+ "event": "finalize",
334
+ "answer": answer,
335
+ "source": source,
336
+ "confidence": confidence,
337
+ },
338
+ )
339
+ return {
340
+ "answer": normalize_answer(answer),
341
+ "source": f"langgraph.{state.get('question_type', 'unknown')}.{source}",
342
+ "confidence": confidence,
343
+ "evidence": self._trace_text(trace),
344
+ "error": final_error,
345
+ "trace": trace,
346
+ }
347
 
348
+ def _run_tool_node(
349
+ self,
350
+ state: GaiaWorkflowState,
351
+ tool_name: str,
352
+ args: dict[str, Any],
353
+ ) -> GaiaWorkflowState:
354
+ observation = execute_tool(tool_name, args, self._context(state))
355
+ return self._state_with_observation(state, observation, tool_name)
356
 
357
+ def _state_with_observation(
358
+ self,
359
+ state: GaiaWorkflowState,
360
+ observation: dict[str, Any],
361
+ event_name: str,
362
+ ) -> GaiaWorkflowState:
363
+ compact_observation = self._compact_observation(observation)
364
+ return {
365
+ "observation": observation,
366
+ "trace": self._append_trace(
367
+ state,
368
  {
369
+ "event": event_name,
 
370
  "observation": compact_observation,
371
+ },
372
+ ),
373
+ }
374
 
375
+ def _run_fallback_tools(self, state: GaiaWorkflowState) -> dict[str, Any]:
376
+ file_name = state.get("file_name", "").lower()
377
+ question = state["question"]
378
+ fallback_steps: list[tuple[str, dict[str, Any]]] = []
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
379
 
380
+ if file_name.endswith(tuple(SPREADSHEET_EXTENSIONS)):
381
+ fallback_steps.append(("spreadsheet_tool", {}))
382
+ if file_name.endswith(".py"):
383
+ fallback_steps.append(("python_tool", {}))
384
+ if file_name.endswith(tuple(TEXT_EXTENSIONS)):
385
+ fallback_steps.append(("attachment_text_tool", {}))
 
 
 
386
 
387
+ fallback_steps.extend(
388
+ [
389
+ ("direct_answer_tool", {}),
390
+ ("sports_tool", {}),
391
+ ("wikipedia_tool", {"query": state.get("type_query") or question}),
392
+ ]
393
  )
394
 
395
+ urls = extract_urls(question)
396
+ if urls:
397
+ if any(is_youtube_url(url) for url in urls):
398
+ fallback_steps.append(("video_tool", {}))
399
+ else:
400
+ return self._read_urls_observation(state)
401
+
402
+ if file_name.endswith(tuple(AUDIO_VIDEO_EXTENSIONS)):
403
+ fallback_steps.append(("audio_tool" if file_name.endswith(".mp3") else "video_tool", {}))
404
+ if file_name.endswith(tuple(IMAGE_EXTENSIONS)):
405
+ fallback_steps.append(("vision_tool", {}))
406
+
407
+ tried = []
408
+ for index, (tool_name, args) in enumerate(fallback_steps, start=1):
409
+ if index > MAX_AGENT_STEPS:
410
+ break
411
+ observation = execute_tool(tool_name, args, self._context(state))
412
+ tried.append(self._compact_observation(observation))
413
+ if self._observation_is_useful(observation):
414
+ observation["fallback_tried"] = tried
415
+ return observation
416
+
417
+ search_observation = self._search_and_read_observation(state)
418
+ search_observation["fallback_tried"] = tried
419
+ return search_observation
420
 
421
+ def _read_urls_observation(self, state: GaiaWorkflowState) -> dict[str, Any]:
422
+ urls = [url for url in extract_urls(state["question"]) if not is_youtube_url(url)]
423
+ if not urls:
424
+ return execute_tool("video_tool", {}, self._context(state))
425
+
426
+ observations = []
427
+ for url in urls[:2]:
428
+ observations.append(execute_tool("web_read_tool", {"url": url}, self._context(state)))
429
+ evidence = "\n\n".join(
430
+ f"URL {index}: {item.get('evidence', '')}"
431
+ for index, item in enumerate(observations, start=1)
432
+ )
433
+ errors = [item.get("error", "") for item in observations if item.get("error")]
434
+ return {
435
+ "tool": "web_url_workflow",
436
+ "ok": any(item.get("ok") for item in observations),
437
+ "answer": None,
438
+ "confidence": "medium" if any(item.get("ok") for item in observations) else "low",
439
+ "source": "web_url_workflow",
440
+ "evidence": truncate_text(evidence, MAX_TOOL_OUTPUT_CHARS),
441
+ "error": self._join_error(*errors),
442
  }
443
+
444
+ def _search_and_read_observation(self, state: GaiaWorkflowState) -> dict[str, Any]:
445
+ query = state.get("type_query") or state["question"]
446
+ search_observation = execute_tool(
447
+ "web_search_tool",
448
+ {"query": query, "max_results": 5},
449
+ self._context(state),
 
450
  )
451
+ evidence_parts = [str(search_observation.get("evidence") or "")]
452
+ errors = [str(search_observation.get("error") or "")]
453
+
454
+ try:
455
+ search_results = json.loads(str(search_observation.get("evidence") or "[]"))
456
+ except json.JSONDecodeError:
457
+ search_results = []
458
+
459
+ for result in search_results[:2]:
460
+ url = result.get("url", "")
461
+ if not url or is_youtube_url(url):
462
+ continue
463
+ read_observation = execute_tool("web_read_tool", {"url": url}, self._context(state))
464
+ evidence_parts.append(
465
+ f"--- {result.get('title', url)} ({url}) ---\n{read_observation.get('evidence', '')}"
466
+ )
467
+ if read_observation.get("error"):
468
+ errors.append(str(read_observation["error"]))
469
+
470
+ return {
471
+ "tool": "web_search_workflow",
472
+ "ok": bool(search_results),
473
+ "answer": None,
474
+ "confidence": "medium" if search_results else "low",
475
+ "source": "web_search_workflow",
476
+ "evidence": truncate_text("\n\n".join(evidence_parts), MAX_TOOL_OUTPUT_CHARS),
477
+ "error": self._join_error(*errors),
478
+ }
479
+
480
+ def _context(self, state: GaiaWorkflowState) -> dict[str, str]:
481
+ return {
482
+ "question": state["question"],
483
+ "task_id": state.get("task_id", ""),
484
+ "file_name": state.get("file_name", ""),
485
+ }
486
+
487
+ def _observation_is_useful(self, observation: dict[str, Any]) -> bool:
488
+ if observation.get("answer"):
489
+ return True
490
+ return bool(observation.get("ok") and observation.get("evidence"))
491
+
492
+ def _append_trace(
493
+ self,
494
+ state: GaiaWorkflowState,
495
+ event: dict[str, Any],
496
+ ) -> list[dict[str, Any]]:
497
+ return list(state.get("trace", [])) + [event]
498
 
499
  def _compact_observation(self, observation: dict[str, Any]) -> dict[str, Any]:
500
  compact = dict(observation)
501
  if compact.get("evidence"):
502
  compact["evidence"] = truncate_text(str(compact["evidence"]), MAX_TOOL_OUTPUT_CHARS)
503
+ if compact.get("fallback_tried"):
504
+ compact["fallback_tried"] = [
505
+ self._compact_observation(item) for item in compact["fallback_tried"]
506
+ ]
507
  return compact
508
 
509
  def _trace_text(self, trace: list[dict[str, Any]]) -> str:
510
  return truncate_text(json.dumps(trace, ensure_ascii=False, indent=2), MAX_TOOL_OUTPUT_CHARS)
511
 
512
+ def _join_error(self, *errors: Any) -> str:
513
+ return "; ".join(str(error) for error in errors if str(error or "").strip())
514
+
515
 
516
  # 兼容原模板里的 BasicAgent 名称。
517
  BasicAgent = GaiaAgent
app.py CHANGED
@@ -5,7 +5,12 @@ import pandas as pd
5
  import requests
6
 
7
  from agent import GaiaAgent
8
- from config import DEFAULT_API_URL, HF_TEXT_MODEL, HF_VISION_MODEL
 
 
 
 
 
9
 
10
 
11
  def run_and_submit_all(profile: gr.OAuthProfile | None):
@@ -150,19 +155,21 @@ def run_and_submit_all(profile: gr.OAuthProfile | None):
150
 
151
 
152
  with gr.Blocks() as demo:
153
- gr.Markdown("# LLM 工具调用 GAIA Agent 评测运行器")
154
  gr.Markdown(
155
  f"""
156
  **当前策略:**
157
 
158
- 1. LLM planner 会输出 JSON action,决定调用哪个工具。
159
- 2. 工具执行器会调用规则、Python、Excel、Wikipedia、体育统计、网页搜索等工具。
160
- 3. 每题最多运行有限步数,避免无限搜索和过度消耗额度。
161
- 4. 音频、视频和棋局图工具当前禁用;LLM 遇到这类题应返回“无法确定”。
162
- 5. 当前文本模型:`{HF_TEXT_MODEL}`;当前视觉模型:`{HF_VISION_MODEL or "未启用"}`。
 
 
163
 
164
  **提交前检查:**
165
- Space Secrets 至少需要 `HF_TOKEN`,因为每道题都需要 LLM planner 决定工具调用。
166
  """
167
  )
168
 
@@ -192,5 +199,5 @@ if __name__ == "__main__":
192
  print("未找到 SPACE_ID 环境变量(可能是在本地运行)。无法确定仓库地址。")
193
 
194
  print("-" * (60 + len(" 应用启动中 ")) + "\n")
195
- print("正在启动 LLM 工具调用 GAIA Agent 评测 Gradio 界面...")
196
  demo.launch(debug=True, share=False)
 
5
  import requests
6
 
7
  from agent import GaiaAgent
8
+ from config import (
9
+ DEFAULT_API_URL,
10
+ HF_PLANNER_USE_RESPONSE_FORMAT,
11
+ HF_TEXT_MODEL,
12
+ HF_VISION_MODEL,
13
+ )
14
 
15
 
16
  def run_and_submit_all(profile: gr.OAuthProfile | None):
 
155
 
156
 
157
  with gr.Blocks() as demo:
158
+ gr.Markdown("# LangGraph GAIA Agent 评测运行器")
159
  gr.Markdown(
160
  f"""
161
  **当前策略:**
162
 
163
+ 1. LangGraph 先调用 LLM 做题型分类,明确应该使用哪类工具。
164
+ 2. 工作流按题型进入规则、Python、Excel、Wikipedia、体育统计、网页读取/搜索等处理节点。
165
+ 3. 工具结果统一交给 LLM 做最终答案格式化;LLM 失败时使用工具候选答案兜底。
166
+ 4. 未识别类型或工具无结果时进入兜底节点,按低成本工具顺序尝试。
167
+ 5. 音频、视频和棋局图工具当前禁用;这类题会返回“无法确定”。
168
+ 6. 当前文本模型:`{HF_TEXT_MODEL}`;当前视觉模型:`{HF_VISION_MODEL or "未启用"}`。
169
+ 7. 强制 JSON response_format:`{HF_PLANNER_USE_RESPONSE_FORMAT}`。
170
 
171
  **提交前检查:**
172
+ Space Secrets 至少需要 `HF_TOKEN`,且该 token 必须允许调用 Inference Providers。
173
  """
174
  )
175
 
 
199
  print("未找到 SPACE_ID 环境变量(可能是在本地运行)。无法确定仓库地址。")
200
 
201
  print("-" * (60 + len(" 应用启动中 ")) + "\n")
202
+ print("正在启动 LangGraph GAIA Agent 评测 Gradio 界面...")
203
  demo.launch(debug=True, share=False)
config.py CHANGED
@@ -7,6 +7,10 @@ DEFAULT_API_URL = "https://agents-course-unit4-scoring.hf.space"
7
  HF_ROUTER_URL = "https://router.huggingface.co/v1/chat/completions"
8
  HF_TEXT_MODEL = os.getenv("HF_TEXT_MODEL", "openai/gpt-oss-20b:cheapest")
9
  HF_VISION_MODEL = os.getenv("HF_VISION_MODEL", "")
 
 
 
 
10
  MAX_EVIDENCE_CHARS = int(os.getenv("MAX_EVIDENCE_CHARS", "18000"))
11
  MAX_AGENT_STEPS = int(os.getenv("MAX_AGENT_STEPS", "4"))
12
  MAX_TOOL_OUTPUT_CHARS = int(os.getenv("MAX_TOOL_OUTPUT_CHARS", "8000"))
 
7
  HF_ROUTER_URL = "https://router.huggingface.co/v1/chat/completions"
8
  HF_TEXT_MODEL = os.getenv("HF_TEXT_MODEL", "openai/gpt-oss-20b:cheapest")
9
  HF_VISION_MODEL = os.getenv("HF_VISION_MODEL", "")
10
+ HF_PLANNER_USE_RESPONSE_FORMAT = (
11
+ os.getenv("HF_PLANNER_USE_RESPONSE_FORMAT", "0").strip().lower()
12
+ in {"1", "true", "yes", "on"}
13
+ )
14
  MAX_EVIDENCE_CHARS = int(os.getenv("MAX_EVIDENCE_CHARS", "18000"))
15
  MAX_AGENT_STEPS = int(os.getenv("MAX_AGENT_STEPS", "4"))
16
  MAX_TOOL_OUTPUT_CHARS = int(os.getenv("MAX_TOOL_OUTPUT_CHARS", "8000"))
requirements.txt CHANGED
@@ -3,3 +3,4 @@ requests
3
  pandas
4
  beautifulsoup4
5
  openpyxl
 
 
3
  pandas
4
  beautifulsoup4
5
  openpyxl
6
+ langgraph
tools/llm_client.py CHANGED
@@ -1,11 +1,17 @@
1
  import json
2
  import re
 
3
  from dataclasses import dataclass
4
  from typing import Any
5
 
6
  import requests
7
 
8
- from config import HF_ROUTER_URL, HF_TEXT_MODEL, get_hf_token
 
 
 
 
 
9
  from tools.common import normalize_answer
10
 
11
 
@@ -22,6 +28,22 @@ class AgentAction:
22
  return self.action == "final_answer"
23
 
24
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
  def call_hf_chat(
26
  messages: list[dict[str, Any]],
27
  model: str = HF_TEXT_MODEL,
@@ -56,27 +78,115 @@ def call_hf_chat(
56
  f"HF chat 请求失败:status={response.status_code}, body={response.text[:500]}"
57
  )
58
  data = response.json()
59
- content = data["choices"][0]["message"].get("content")
60
- if content is None:
61
- return ""
62
- return content
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
63
 
64
 
65
  def extract_json_object(text: str) -> dict[str, Any]:
66
  text = text.strip()
67
- if text.startswith("```"):
68
- text = re.sub(r"^```(?:json)?", "", text, flags=re.IGNORECASE).strip()
69
- text = re.sub(r"```$", "", text).strip()
70
 
71
- try:
72
- return json.loads(text)
73
- except json.JSONDecodeError:
74
- pass
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75
 
76
- match = re.search(r"\{.*\}", text, flags=re.DOTALL)
77
- if not match:
78
- raise ValueError(f"模型没有输出 JSON 对象:{text[:300]}")
79
- return json.loads(match.group(0))
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
80
 
81
 
82
  def parse_agent_action(text: str) -> AgentAction:
@@ -102,7 +212,149 @@ def parse_agent_action(text: str) -> AgentAction:
102
  )
103
 
104
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
105
  def call_planner_model(messages: list[dict[str, Any]]) -> str:
 
 
 
 
 
 
 
106
  try:
107
  return call_hf_chat(
108
  messages,
@@ -111,8 +363,7 @@ def call_planner_model(messages: list[dict[str, Any]]) -> str:
111
  response_format={"type": "json_object"},
112
  )
113
  except RuntimeError as exc:
114
- error_text = str(exc).lower()
115
- if "response_format" not in error_text and "json_object" not in error_text:
116
  raise
117
  return call_hf_chat(
118
  messages,
@@ -121,6 +372,20 @@ def call_planner_model(messages: list[dict[str, Any]]) -> str:
121
  )
122
 
123
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
124
  def plan_next_action(messages: list[dict[str, Any]]) -> AgentAction:
125
  raw = call_planner_model(messages)
126
  try:
 
1
  import json
2
  import re
3
+ import ast
4
  from dataclasses import dataclass
5
  from typing import Any
6
 
7
  import requests
8
 
9
+ from config import (
10
+ HF_PLANNER_USE_RESPONSE_FORMAT,
11
+ HF_ROUTER_URL,
12
+ HF_TEXT_MODEL,
13
+ get_hf_token,
14
+ )
15
  from tools.common import normalize_answer
16
 
17
 
 
28
  return self.action == "final_answer"
29
 
30
 
31
+ @dataclass
32
+ class QuestionClassification:
33
+ question_type: str
34
+ confidence: str = "medium"
35
+ reason: str = ""
36
+ query: str = ""
37
+ raw: str = ""
38
+
39
+
40
+ @dataclass
41
+ class FinalAnswerFormat:
42
+ answer: str
43
+ confidence: str = "medium"
44
+ raw: str = ""
45
+
46
+
47
  def call_hf_chat(
48
  messages: list[dict[str, Any]],
49
  model: str = HF_TEXT_MODEL,
 
78
  f"HF chat 请求失败:status={response.status_code}, body={response.text[:500]}"
79
  )
80
  data = response.json()
81
+ try:
82
+ message = data["choices"][0]["message"]
83
+ except (KeyError, IndexError, TypeError) as exc:
84
+ raise RuntimeError(f"HF chat 响应结构异常:{_compact_json(data)}") from exc
85
+ return extract_message_text(message, data)
86
+
87
+
88
+ def extract_message_text(message: dict[str, Any], raw_response: dict[str, Any]) -> str:
89
+ """兼容不同 HF Router provider 返回的 chat message content 格式。"""
90
+ content = message.get("content")
91
+ if isinstance(content, str) and content.strip():
92
+ return content
93
+ if isinstance(content, list):
94
+ parts = []
95
+ for item in content:
96
+ if isinstance(item, str):
97
+ parts.append(item)
98
+ elif isinstance(item, dict):
99
+ for key in ("text", "content"):
100
+ value = item.get(key)
101
+ if isinstance(value, str):
102
+ parts.append(value)
103
+ joined = "\n".join(part for part in parts if part.strip()).strip()
104
+ if joined:
105
+ return joined
106
+
107
+ fallback_parts = []
108
+ for key in ("reasoning_content", "reasoning", "text"):
109
+ value = message.get(key)
110
+ if isinstance(value, str) and value.strip():
111
+ fallback_parts.append(f"{key}={value}")
112
+ if fallback_parts:
113
+ return "\n".join(fallback_parts)
114
+
115
+ raise RuntimeError(
116
+ "HF chat 返回空内容。"
117
+ f" message_keys={sorted(message.keys())}; raw={_compact_json(raw_response)}"
118
+ )
119
 
120
 
121
  def extract_json_object(text: str) -> dict[str, Any]:
122
  text = text.strip()
123
+ candidates = [text]
 
 
124
 
125
+ fenced_blocks = re.findall(
126
+ r"```(?:json)?\s*(.*?)```",
127
+ text,
128
+ flags=re.IGNORECASE | re.DOTALL,
129
+ )
130
+ candidates.extend(block.strip() for block in fenced_blocks)
131
+ candidates.extend(_balanced_json_candidates(text))
132
+
133
+ errors = []
134
+ for candidate in candidates:
135
+ if not candidate:
136
+ continue
137
+ try:
138
+ parsed = json.loads(candidate)
139
+ except json.JSONDecodeError as exc:
140
+ errors.append(str(exc))
141
+ else:
142
+ if isinstance(parsed, dict):
143
+ return parsed
144
+
145
+ try:
146
+ parsed_literal = ast.literal_eval(candidate)
147
+ except (SyntaxError, ValueError):
148
+ continue
149
+ if isinstance(parsed_literal, dict):
150
+ return parsed_literal
151
 
152
+ error_hint = f"; parse_errors={errors[:2]}" if errors else ""
153
+ raise ValueError(f"模型没有输出 JSON 对象:{text[:300]}{error_hint}")
154
+
155
+
156
+ def _balanced_json_candidates(text: str) -> list[str]:
157
+ candidates = []
158
+ starts = [index for index, char in enumerate(text) if char == "{"]
159
+ for start in starts:
160
+ depth = 0
161
+ in_string = False
162
+ escaped = False
163
+ for index in range(start, len(text)):
164
+ char = text[index]
165
+ if in_string:
166
+ if escaped:
167
+ escaped = False
168
+ elif char == "\\":
169
+ escaped = True
170
+ elif char == '"':
171
+ in_string = False
172
+ continue
173
+ if char == '"':
174
+ in_string = True
175
+ elif char == "{":
176
+ depth += 1
177
+ elif char == "}":
178
+ depth -= 1
179
+ if depth == 0:
180
+ candidates.append(text[start : index + 1])
181
+ break
182
+ return candidates
183
+
184
+
185
+ def _compact_json(data: Any, limit: int = 1200) -> str:
186
+ text = json.dumps(data, ensure_ascii=False, default=str)
187
+ if len(text) <= limit:
188
+ return text
189
+ return text[:limit] + "...<truncated>"
190
 
191
 
192
  def parse_agent_action(text: str) -> AgentAction:
 
212
  )
213
 
214
 
215
+ def classify_question_type(
216
+ question: str,
217
+ task_id: str,
218
+ file_name: str,
219
+ type_specs: list[dict[str, str]],
220
+ ) -> QuestionClassification:
221
+ allowed_types = {spec["type"] for spec in type_specs}
222
+ type_lines = "\n".join(
223
+ (
224
+ f"- {spec['type']}: tool={spec['tool']}; "
225
+ f"description={spec['description']}"
226
+ )
227
+ for spec in type_specs
228
+ )
229
+ prompt = f"""
230
+ Classify this GAIA task into exactly one question_type.
231
+
232
+ Available question types:
233
+ {type_lines}
234
+
235
+ Return exactly one JSON object:
236
+ {{
237
+ "question_type": "one allowed type",
238
+ "confidence": "high|medium|low",
239
+ "reason": "short reason",
240
+ "query": "optional search/query string if useful"
241
+ }}
242
+
243
+ Rules:
244
+ 1. Do not answer the question.
245
+ 2. Choose the tool category that should be tried first.
246
+ 3. If no type clearly fits, use "unknown".
247
+ 4. Prefer direct_text for self-contained table, string, list, or regex-like tasks.
248
+ 5. Prefer python_code for .py attachments.
249
+ 6. Prefer spreadsheet for .xlsx/.xls attachments.
250
+
251
+ Task:
252
+ task_id={task_id}
253
+ file_name={file_name}
254
+ question={question}
255
+ """.strip()
256
+
257
+ raw = call_hf_chat(
258
+ [
259
+ {
260
+ "role": "system",
261
+ "content": (
262
+ "You are a strict GAIA task classifier. "
263
+ "Return JSON only. Never solve the task."
264
+ ),
265
+ },
266
+ {"role": "user", "content": prompt},
267
+ ],
268
+ max_tokens=350,
269
+ temperature=0.0,
270
+ )
271
+ data = extract_json_object(raw)
272
+ question_type = str(data.get("question_type") or data.get("type") or "unknown").strip()
273
+ if question_type not in allowed_types:
274
+ question_type = "unknown"
275
+ confidence = str(data.get("confidence", "medium")).strip() or "medium"
276
+ reason = str(data.get("reason", "")).strip()
277
+ query = str(data.get("query", "")).strip()
278
+ return QuestionClassification(
279
+ question_type=question_type,
280
+ confidence=confidence,
281
+ reason=reason,
282
+ query=query,
283
+ raw=raw,
284
+ )
285
+
286
+
287
+ def format_final_answer_with_llm(
288
+ question: str,
289
+ question_type: str,
290
+ candidate_answer: str,
291
+ evidence: str,
292
+ source: str,
293
+ confidence: str,
294
+ ) -> FinalAnswerFormat:
295
+ prompt = f"""
296
+ Format the tool result into the exact final answer required by the GAIA question.
297
+
298
+ Question:
299
+ {question}
300
+
301
+ Question type:
302
+ {question_type}
303
+
304
+ Tool source:
305
+ {source}
306
+
307
+ Tool confidence:
308
+ {confidence}
309
+
310
+ Candidate answer:
311
+ {candidate_answer or "<none>"}
312
+
313
+ Tool evidence:
314
+ {evidence or "<none>"}
315
+
316
+ Return exactly one JSON object:
317
+ {{"answer":"final answer only","confidence":"high|medium|low"}}
318
+
319
+ Rules:
320
+ 1. Do not explain.
321
+ 2. If candidate_answer is present, preserve its factual content and only fix formatting.
322
+ 3. Obey requested capitalization, decimal places, ordering, separators, and units.
323
+ 4. If evidence is insufficient and no candidate_answer is available, return answer "无法确定" with low confidence.
324
+ """.strip()
325
+ raw = call_hf_chat(
326
+ [
327
+ {
328
+ "role": "system",
329
+ "content": (
330
+ "You are a strict final-answer formatter. "
331
+ "Return JSON only and never include reasoning."
332
+ ),
333
+ },
334
+ {"role": "user", "content": prompt},
335
+ ],
336
+ max_tokens=300,
337
+ temperature=0.0,
338
+ )
339
+ data = extract_json_object(raw)
340
+ answer = data.get("answer") or data.get("final_answer")
341
+ if answer is None:
342
+ raise ValueError(f"格式化模型 JSON 缺少 answer 字段:{data}")
343
+ return FinalAnswerFormat(
344
+ answer=normalize_answer(str(answer)),
345
+ confidence=str(data.get("confidence", confidence)).strip() or confidence,
346
+ raw=raw,
347
+ )
348
+
349
+
350
  def call_planner_model(messages: list[dict[str, Any]]) -> str:
351
+ if not HF_PLANNER_USE_RESPONSE_FORMAT:
352
+ return call_hf_chat(
353
+ messages,
354
+ max_tokens=700,
355
+ temperature=0.0,
356
+ )
357
+
358
  try:
359
  return call_hf_chat(
360
  messages,
 
363
  response_format={"type": "json_object"},
364
  )
365
  except RuntimeError as exc:
366
+ if not _should_retry_without_response_format(str(exc)):
 
367
  raise
368
  return call_hf_chat(
369
  messages,
 
372
  )
373
 
374
 
375
+ def _should_retry_without_response_format(error_text: str) -> bool:
376
+ lower_error = error_text.lower()
377
+ return any(
378
+ marker in lower_error
379
+ for marker in (
380
+ "response_format",
381
+ "json_object",
382
+ "json mode",
383
+ "空内容",
384
+ "empty",
385
+ )
386
+ )
387
+
388
+
389
  def plan_next_action(messages: list[dict[str, Any]]) -> AgentAction:
390
  raw = call_planner_model(messages)
391
  try: