VibecoderMcSwaggins commited on
Commit
b9709ff
Β·
1 Parent(s): 8abea0c

docs: Deep technical analysis of P2 duplicate report bug

Browse files

Comprehensive bug report with:
- Executive summary confirming this is a stack bug (not model)
- Microsoft Agent Framework event type reference
- Event flow trace showing exact duplication path
- Key code paths with line numbers
- Buffer clearing analysis (why comparison isn't possible)
- Edge case analysis
- Verification checklist

Root cause: MagenticFinalResultEvent/WorkflowOutputEvent contains
same content that was already streamed, and no deduplication exists.

docs/bugs/P2_DUPLICATE_REPORT_CONTENT.md CHANGED
@@ -4,6 +4,18 @@
4
  **Status**: OPEN
5
  **Severity**: P2 (UX - Duplicate content confuses users)
6
  **Component**: `src/orchestrators/advanced.py` + `src/app.py`
 
 
 
 
 
 
 
 
 
 
 
 
7
 
8
  ---
9
 
@@ -215,7 +227,116 @@ if has_substantial_streaming:
215
 
216
  ---
217
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
218
  ## Related
219
 
220
  - **Not related to model quality** - This is a stack bug, not model limitation
221
  - P1 Free Tier fix (PR fix/P1-free-tier) enabled streaming, exposing this bug
 
 
4
  **Status**: OPEN
5
  **Severity**: P2 (UX - Duplicate content confuses users)
6
  **Component**: `src/orchestrators/advanced.py` + `src/app.py`
7
+ **Affects**: Both Free Tier (HuggingFace) AND Paid Tier (OpenAI)
8
+
9
+ ---
10
+
11
+ ## Executive Summary
12
+
13
+ This is a **confirmed stack bug**, NOT a model limitation. The duplicate report appears because:
14
+
15
+ 1. Streaming events yield the full report content character-by-character
16
+ 2. Final events (`MagenticFinalResultEvent`/`WorkflowOutputEvent`) contain the SAME content
17
+ 3. No deduplication exists between streamed content and final event content
18
+ 4. Both are appended to the output
19
 
20
  ---
21
 
 
227
 
228
  ---
229
 
230
+ ## Deep Technical Analysis
231
+
232
+ ### Microsoft Agent Framework Event Types
233
+
234
+ The framework emits these event types (all inherit from `WorkflowEvent`):
235
+
236
+ | Event Type | Purpose | Key Attributes |
237
+ |------------|---------|----------------|
238
+ | `MagenticAgentDeltaEvent` | Streaming tokens | `text`, `agent_id` |
239
+ | `MagenticAgentMessageEvent` | Agent turn complete | `message` (ChatMessage), `agent_id` |
240
+ | `MagenticFinalResultEvent` | Workflow final result | `message` (ChatMessage) |
241
+ | `MagenticOrchestratorMessageEvent` | Manager bookkeeping | `message`, `kind`, `orchestrator_id` |
242
+ | `WorkflowOutputEvent` | Workflow output | `data`, `source_executor_id` |
243
+
244
+ ### Event Flow Trace
245
+
246
+ ```
247
+ PHASE 1: Agent Streaming (Reporter)
248
+ ─────────────────────────────────────
249
+ MagenticAgentDeltaEvent(text="##", agent_id="reporter") β†’ yields streaming event
250
+ MagenticAgentDeltaEvent(text=" Summary", agent_id="reporter") β†’ yields streaming event
251
+ MagenticAgentDeltaEvent(text="\n", agent_id="reporter") β†’ yields streaming event
252
+ ... (hundreds more delta events)
253
+ MagenticAgentDeltaEvent(text=".", agent_id="reporter") β†’ yields streaming event
254
+
255
+ β†’ Result: Full report content in streaming_buffer (app.py) and current_message_buffer (orchestrator)
256
+
257
+ PHASE 2: Agent Completion
258
+ ─────────────────────────────────────
259
+ MagenticAgentMessageEvent(message=ChatMessage(...), agent_id="reporter")
260
+ β†’ _handle_completion_event() yields: "reporter: [first 200 chars]..."
261
+ β†’ Clears current_message_buffer
262
+ β†’ app.py flushes streaming_buffer to response_parts with "πŸ“‘ **STREAMING**:" prefix
263
+
264
+ PHASE 3: Workflow Termination (THE BUG)
265
+ ─────────────────────────────────────
266
+ MagenticFinalResultEvent(message=ChatMessage(...)) ← Contains SAME full report!
267
+ OR
268
+ WorkflowOutputEvent(data=ChatMessage(...)) ← Contains SAME full report!
269
+
270
+ β†’ _process_event() extracts text with _extract_text()
271
+ β†’ Returns AgentEvent(type="complete", message=FULL_REPORT)
272
+ β†’ app.py appends FULL_REPORT to response_parts (NO prefix)
273
+
274
+ RESULT: Report appears twice:
275
+ 1. "πŸ“‘ **STREAMING**: [full report]"
276
+ 2. "[full report again]"
277
+ ```
278
+
279
+ ### Key Code Paths
280
+
281
+ **`advanced.py` lines 299-345 (main loop):**
282
+ ```python
283
+ # Buffer is cleared HERE (line 337) after MagenticAgentMessageEvent
284
+ current_message_buffer = ""
285
+
286
+ # But MagenticFinalResultEvent comes AFTER and _process_event has no buffer context!
287
+ agent_event = self._process_event(event, iteration) # line 341
288
+ if agent_event:
289
+ yield agent_event # line 345 - yields duplicate!
290
+ ```
291
+
292
+ **`advanced.py` lines 532-539 (_process_event):**
293
+ ```python
294
+ elif isinstance(event, MagenticFinalResultEvent):
295
+ text = self._extract_text(event.message) # Extracts FULL content
296
+ return AgentEvent(type="complete", message=text) # Returns FULL content
297
+ ```
298
+
299
+ **`app.py` lines 229-232 (UI handling):**
300
+ ```python
301
+ if event.type == "complete":
302
+ response_parts.append(event.message) # Appends to existing streamed content!
303
+ yield "\n\n".join(response_parts)
304
+ ```
305
+
306
+ ### Why Buffer Clearing Doesn't Help
307
+
308
+ The `current_message_buffer` is cleared (line 337) BEFORE the final events arrive. So even if we wanted to compare, we've already lost the reference:
309
+
310
+ ```python
311
+ # Line 327-338: Handle MagenticAgentMessageEvent
312
+ iteration += 1
313
+ comp_event, prog_event = self._handle_completion_event(...)
314
+ yield comp_event
315
+ yield prog_event
316
+ current_message_buffer = "" # CLEARED!
317
+ continue
318
+
319
+ # Line 341-345: Handle final events (buffer is empty now!)
320
+ agent_event = self._process_event(event, iteration) # No buffer context
321
+ ```
322
+
323
+ ### Potential Edge Cases
324
+
325
+ 1. **Tool-only turns**: If agent makes tool calls without text, buffer is empty β†’ fallback text used
326
+ 2. **Multiple agents streaming**: Buffer clears on agent switch (line 311-313) β†’ OK
327
+ 3. **Timeout**: Uses `_handle_timeout()` which invokes ReportAgent directly β†’ Different path
328
+ 4. **No final event**: Falls back to "Research completed..." message (line 354-363) β†’ OK
329
+
330
+ ### Verification Needed
331
+
332
+ - [ ] Confirm `MagenticFinalResultEvent` vs `WorkflowOutputEvent` - which is emitted?
333
+ - [ ] Confirm bug occurs on both Free and Paid tiers
334
+ - [ ] Measure content length match between streaming and final event
335
+
336
+ ---
337
+
338
  ## Related
339
 
340
  - **Not related to model quality** - This is a stack bug, not model limitation
341
  - P1 Free Tier fix (PR fix/P1-free-tier) enabled streaming, exposing this bug
342
+ - SPEC-17 Accumulator Pattern addressed repr bug but created this side effect