Fixed THREE critical issues:

Issue 1: Content Truncation ✅ FIXED

- Problem: Only 200 chars preview returned
- Fix: Modified confluence_loader to return full content

Issue 2: Wrong Pages Loaded ✅ FIXED

- Problem: Tool loaded random 3 pages, might miss the target page
- Fix: Added page_title parameter for specific page filtering

Issue 3: HTML Processor Bug ✅ FIXED (This was the root cause
- Problem: Buggy _extract_structured_text method only extracted 77 chars out of 49,319
- Fix: Rewrote to use recursive processing instead of flawed soup.descendants iteration
- Result: Now extracts 13,360 chars (173x improvement)

app_with_confluence.py

@@ -370,9 +370,12 @@ When answering questions:
 **CRITICAL: How to Handle Confluence Information Requests**
 
 When users ask you to "report", "list", "find", "show", or "provide" specific information from documents:
-1. **FIRST**: Use the confluence_search tool to find the relevant document
-2. **SECOND**: Use the confluence_loader tool to load the FULL page content
-3. **THIRD**: Extract and present the requested information directly in your response
+1. **FIRST**: Use the confluence_search tool to find the relevant document and identify its title
+2. **SECOND**: Use the confluence_loader tool with BOTH space_key AND page_title parameters to load that specific page
+   - Example: `confluence_loader(space_key="Acquisitio", page_title="fraud-model-governance")`
+   - ALWAYS provide the page_title parameter when you know which specific page to load
+   - This ensures you get the exact page content, not random pages from the space
+3. **THIRD**: Extract and present the requested information directly in your response from the loaded content
 4. **FOURTH**: Provide the Confluence page citation/link as a reference for verification
 
 **DO NOT** simply provide a link to a page without extracting the information first.
@@ -380,11 +383,19 @@ The user needs the actual information in your response, not just a pointer to wh
 
 **Example of CORRECT behavior:**
 - User asks: "Report models and corresponding versions found in the fraud-model-governance document"
-- You should: Search for the document → Load the page → Extract the model inventory table → Present the models and versions in a formatted list → Provide the page link for reference
+- You should:
+  1. Search: `confluence_search("fraud model governance")` → finds page titled "fraud-model-governance"
+  2. Load: `confluence_loader(space_key="Acquisitio", page_title="fraud-model-governance")` → loads FULL page
+  3. Extract the model inventory table from the loaded content
+  4. Present the models and versions in a formatted list
+  5. Provide the page link for reference
 
 **Example of INCORRECT behavior:**
 - User asks: "Report models and corresponding versions found in the fraud-model-governance document"
-- You should NOT: Search for the document → Provide only a link to the page without extracting the information
+- You should NOT:
+  - Search for the document → Provide only a link without extracting
+  - Load random pages without specifying page_title → Get wrong pages
+  - Hallucinate model names that aren't in the actual document
 
 **When to Use Confluence Tools:**
 - **For compliance questions**: ALWAYS search for relevant policy documentation, extract key policies, then cite sources
@@ -426,7 +437,7 @@ def create_enhanced_agent_phase3():
 
         # Create Confluence tools
         search_confluence = create_confluence_search_tool(rag=rag, k=5)
-        load_confluence_page = create_confluence_loader_tool(max_pages=3)
+        load_confluence_page = create_confluence_loader_tool(max_pages=10)  # Increased to ensure all pages available for filtering
 
         # Add all tools
         tools = [