VibecoderMcSwaggins commited on
Commit
ecaa2e8
Β·
1 Parent(s): 8c0ec2b

fix(P0): Complete bi-directional tool support for HuggingFace (sync+streaming)

Browse files
docs/bugs/ACTIVE_BUGS.md CHANGED
@@ -7,81 +7,24 @@
7
 
8
  ## P0 - Critical
9
 
10
- ### P0 - AIFunction Not JSON Serializable (Free Tier Broken)
11
- **File:** `docs/bugs/P0_AIFUNCTION_NOT_JSON_SERIALIZABLE.md`
12
- **Found:** 2025-12-01 (HuggingFace Spaces)
13
-
14
- **Problem:** Every search round fails with "Object of type AIFunction is not JSON serializable".
15
-
16
- **Error:**
17
- ```
18
- πŸ“š SEARCH_COMPLETE: searcher: Agent searcher: Error processing request -
19
- Object of type AIFunction is not JSON serializable
20
- ```
21
-
22
- **Root Cause:** `HuggingFaceChatClient` passes raw `AIFunction` objects to `InferenceClient.chat_completion()`. When `requests` tries to serialize them to JSON, it fails.
23
-
24
- **Impact:** Free Tier cannot do any research. 5 rounds of errors, no results.
25
-
26
- **Proposed Fix:** Either:
27
- 1. **Quick**: Disable tools with `tools=None` (agents use natural language)
28
- 2. **Proper**: Convert `AIFunction` to JSON schema before passing to HF API
29
 
30
  ---
31
 
32
  ## P3 - UX Polish
 
 
33
 
34
- ### P3 - Progress Bar Positioning/Overlap in ChatInterface
35
- **File:** `docs/bugs/P3_PROGRESS_BAR_POSITIONING.md`
36
- **Found:** 2025-12-01 (HuggingFace Spaces)
37
-
38
- **Problem:** `gr.Progress()` bar renders in strange position inside ChatInterface - floats mid-chat, overlaps text.
39
-
40
- **Root Cause:** Mixing two progress mechanisms:
41
- 1. `gr.Progress()` - general purpose, not designed for ChatInterface
42
- 2. `ChatInterface.show_progress` - built-in chat progress
43
-
44
- **Recommended Fix:** Remove `gr.Progress()`, rely on emoji status text we already emit. Low priority - UX polish only.
45
-
46
- ---
47
-
48
- ## P2 - UX Friction
49
-
50
- ### P2 - Advanced Mode Cold Start Has No User Feedback (βœ… FIXED)
51
- **File:** `docs/bugs/P2_ADVANCED_MODE_COLD_START_NO_FEEDBACK.md`
52
- **Issue:** [#108](https://github.com/The-Obstacle-Is-The-Way/DeepBoner/issues/108)
53
- **Found:** 2025-12-01 (Gradio Testing)
54
-
55
- **Problem:** Three "dead zones" with no visual feedback during Advanced Mode startup:
56
- 1. **Dead Zone #1** (5-15s): Between STARTED β†’ THINKING βœ… FIXED (granular events)
57
- 2. **Dead Zone #2** (10-30s): Between THINKING β†’ PROGRESS (first LLM call) βœ… FIXED (Progress Bar)
58
- 3. **Dead Zone #3** (30-90s): After PROGRESS (SearchAgent executing) βœ… FIXED (Pre-warming + Progress Bar)
59
-
60
- **Phase 1 Fix (commit dbf888c):**
61
- - Added granular progress events during initialization
62
- - Users now see "Loading embedding service...", "Initializing research memory...", "Building agent team..."
63
- - Significantly improves perceived responsiveness
64
-
65
- **Phase 2/3 Fix (Latest):**
66
- - Implemented service pre-warming (`service_loader.warmup_services`)
67
- - Added native Gradio progress bar (`gr.Progress`) to `research_agent`
68
- - Visual feedback is now continuous throughout the entire lifecycle
69
-
70
- ---
71
-
72
- ## P1 - Important
73
-
74
- ### P1 - Memory Layer Not Integrated (Post-Hackathon)
75
- **Issue:** [#73](https://github.com/The-Obstacle-Is-The-Way/DeepBoner/issues/73)
76
- **Spec:** [SPEC_08_INTEGRATE_MEMORY_LAYER.md](../specs/SPEC_08_INTEGRATE_MEMORY_LAYER.md)
77
-
78
- **Problem:** Structured memory (hypotheses, conflicts) is isolated in "God Mode" only.
79
- **Solution:** Extract memory into shared service, integrate into Simple and Advanced modes.
80
- **Status:** Spec written. Blocked until post-hackathon.
81
-
82
- ---
83
 
84
- ## Resolved Bugs
 
 
 
 
85
 
86
  ### ~~P1 - HuggingFace Router 401 Unauthorized~~ FIXED
87
  **File:** `docs/bugs/P1_HUGGINGFACE_ROUTER_401_HYPERBOLIC.md`
 
7
 
8
  ## P0 - Critical
9
 
10
+ (No active P0 bugs)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
 
12
  ---
13
 
14
  ## P3 - UX Polish
15
+ ...
16
+ ## Resolved Bugs
17
 
18
+ ### ~~P0 - AIFunction Not JSON Serializable~~ FIXED
19
+ **File:** `docs/bugs/P0_AIFUNCTION_NOT_JSON_SERIALIZABLE.md`
20
+ **Found:** 2025-12-01
21
+ **Resolved:** 2025-12-01
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
 
23
+ - Problem: `HuggingFaceChatClient` crashed with "Object of type AIFunction is not JSON serializable".
24
+ - Fix: Implemented full bi-directional tool support:
25
+ 1. **Serialization**: Added `_convert_tools` (AIFunction β†’ OpenAI JSON)
26
+ 2. **Parsing (Sync/Async)**: Added `_parse_tool_calls` and streaming accumulator
27
+ - Result: Free Tier now supports full function calling capabilities with Qwen2.5-72B.
28
 
29
  ### ~~P1 - HuggingFace Router 401 Unauthorized~~ FIXED
30
  **File:** `docs/bugs/P1_HUGGINGFACE_ROUTER_401_HYPERBOLIC.md`
docs/bugs/P0_AIFUNCTION_NOT_JSON_SERIALIZABLE.md CHANGED
@@ -1,8 +1,9 @@
1
  # P0 Bug: AIFunction Not JSON Serializable (Free Tier Broken)
2
 
3
  **Severity**: P0 (Critical) - Free Tier cannot perform research
4
- **Status**: In Progress
5
  **Discovered**: 2025-12-01
 
6
  **Reporter**: Production user via HuggingFace Spaces
7
 
8
  ## Symptom
@@ -201,6 +202,22 @@ asyncio.run(test())
201
  # Expected AFTER fix: Research completes with tool calls working
202
  ```
203
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
204
  ## References
205
 
206
  - [HuggingFace Chat Completion - Function Calling](https://huggingface.co/docs/inference-providers/tasks/chat-completion)
 
1
  # P0 Bug: AIFunction Not JSON Serializable (Free Tier Broken)
2
 
3
  **Severity**: P0 (Critical) - Free Tier cannot perform research
4
+ **Status**: RESOLVED
5
  **Discovered**: 2025-12-01
6
+ **Resolved**: 2025-12-01
7
  **Reporter**: Production user via HuggingFace Spaces
8
 
9
  ## Symptom
 
202
  # Expected AFTER fix: Research completes with tool calls working
203
  ```
204
 
205
+ ## Resolution
206
+
207
+ Implemented full function calling support for HuggingFace client:
208
+
209
+ 1. **Request Serialization**: Added `_convert_tools` to map `AIFunction` schemas to OpenAI-compatible JSON.
210
+ 2. **Response Parsing (Sync)**: Added `_parse_tool_calls` to convert HF `tool_calls` to `FunctionCallContent`.
211
+ 3. **Response Parsing (Async)**: Implemented tool call accumulator in `_inner_get_streaming_response` to handle partial tool call deltas and yield valid `FunctionCallContent` objects.
212
+
213
+ ## Verification
214
+
215
+ Verified with unit tests and manual simulation:
216
+
217
+ 1. **Serialization**: Confirmed `AIFunction` -> JSON conversion works for `search_pubmed`.
218
+ 2. **Streaming**: Verified that fragmented tool call deltas (e.g., `{"query":` then `"testosterone"}`) are correctly reassembled into a single `FunctionCallContent`.
219
+ 3. **Integration**: Passed project-level `make check`.
220
+
221
  ## References
222
 
223
  - [HuggingFace Chat Completion - Function Calling](https://huggingface.co/docs/inference-providers/tasks/chat-completion)
src/clients/huggingface.py CHANGED
@@ -240,6 +240,10 @@ class HuggingFaceChatClient(BaseChatClient): # type: ignore[misc]
240
 
241
  stream = await asyncio.to_thread(call_fn)
242
 
 
 
 
 
243
  for chunk in stream:
244
  # Chunk is ChatCompletionStreamOutput
245
  if not chunk.choices:
@@ -247,15 +251,56 @@ class HuggingFaceChatClient(BaseChatClient): # type: ignore[misc]
247
  choice = chunk.choices[0]
248
  delta = choice.delta
249
 
250
- # Convert to ChatResponseUpdate
251
- yield ChatResponseUpdate(
252
- role=cast(Any, delta.role) if delta.role else None,
253
- content=delta.content,
254
- )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
255
 
256
  # Yield control to event loop
257
  await asyncio.sleep(0)
258
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
259
  except Exception as e:
260
  logger.error("HuggingFace Streaming error", error=str(e))
261
  raise
 
240
 
241
  stream = await asyncio.to_thread(call_fn)
242
 
243
+ # Accumulator for tool calls (index -> dict)
244
+ # We need to accumulate because deltas are partial
245
+ tool_call_accumulator: dict[int, dict[str, Any]] = {}
246
+
247
  for chunk in stream:
248
  # Chunk is ChatCompletionStreamOutput
249
  if not chunk.choices:
 
251
  choice = chunk.choices[0]
252
  delta = choice.delta
253
 
254
+ # 1. Handle Text Content
255
+ if delta.content:
256
+ yield ChatResponseUpdate(
257
+ role=cast(Any, delta.role) if delta.role else None,
258
+ text=delta.content,
259
+ )
260
+
261
+ # 2. Handle Tool Calls (Accumulate)
262
+ if hasattr(delta, "tool_calls") and delta.tool_calls:
263
+ for tc in delta.tool_calls:
264
+ idx = tc.index
265
+ if idx not in tool_call_accumulator:
266
+ tool_call_accumulator[idx] = {
267
+ "id": "",
268
+ "name": "",
269
+ "arguments": "",
270
+ }
271
+
272
+ # Merge fields
273
+ if tc.id:
274
+ tool_call_accumulator[idx]["id"] += tc.id
275
+ if tc.function:
276
+ if tc.function.name:
277
+ tool_call_accumulator[idx]["name"] += tc.function.name
278
+ if tc.function.arguments:
279
+ tool_call_accumulator[idx]["arguments"] += tc.function.arguments
280
 
281
  # Yield control to event loop
282
  await asyncio.sleep(0)
283
 
284
+ # 3. Yield Accumulated Tool Calls
285
+ if tool_call_accumulator:
286
+ contents: list[FunctionCallContent] = []
287
+ for idx in sorted(tool_call_accumulator.keys()):
288
+ tc_data = tool_call_accumulator[idx]
289
+ # Only yield if ID and Name are present (required by FunctionCallContent)
290
+ if tc_data["id"] and tc_data["name"]:
291
+ contents.append(
292
+ FunctionCallContent(
293
+ call_id=tc_data["id"],
294
+ name=tc_data["name"],
295
+ arguments=tc_data["arguments"],
296
+ )
297
+ )
298
+
299
+ if contents:
300
+ yield ChatResponseUpdate(
301
+ contents=contents,
302
+ )
303
+
304
  except Exception as e:
305
  logger.error("HuggingFace Streaming error", error=str(e))
306
  raise