INTERACTIVE_FEEDBACK_README.md

Interactive Feedback System for /notes-llm

Overview

The Interactive Feedback System extends the existing Generator-Validator pattern for /notes-llm with user-driven iterative improvements. Users can provide feedback on generated notes, which creates UDF (User Defined Functions) for continuous improvement.

Architecture

Core Components

1. InteractiveFeedbackManager: Manages feedback sessions and UDF generation
2. FeedbackData: Stores user feedback with metadata
3. InteractiveSession: Tracks complete feedback session lifecycle
4. UDF Generation: Creates functions based on user feedback

Data Flow

User Upload → Initial Generation → Interactive Session Created
       ↓
User Feedback → UDF Generation → Feedback Archived
       ↓
Generate with Feedback → Apply UDFs → New Notes with Changes
       ↓
More Feedback or Approval → Continue Loop or Finalize

API Endpoints

1. Initial Generation (Enhanced)

POST /notes-llm

• Purpose: Generate initial notes and create interactive session
• Response Headers:
  • X-Session-ID: Unique session identifier
  • X-Interactive-Enabled: "true"
  • Standard generation metadata

2. Submit Feedback

POST /notes-llm/feedback

• Parameters:
  • session_id: Session identifier
  • feedback_text: User's feedback content
  • feedback_type: "text", "numeric", "formula", "suggestion"
• Response: UDF version and iteration number

3. Generate with Feedback

POST /notes-llm/generate

• Parameters:
  • session_id: Session identifier
  • file: Updated trial balance file
• Response: Notes Excel with applied feedback

4. Approve Session

POST /notes-llm/approve

• Purpose: Finalize session and set final UDF
• Response: Approval confirmation with final UDF

5. Get Session Info

GET /notes-llm/session/{session_id}

• Purpose: Retrieve session status and feedback history
• Response: Complete session information

Feedback Types

1. Text Feedback

• Natural language suggestions
• Example: "Add more detail to the depreciation notes"

2. Numeric Feedback

• Specific numerical adjustments
• Example: "Increase depreciation rate to 15%"

3. Formula Feedback

• Excel formulas or calculation suggestions
• Example: "Use formula: =SUM(Depreciation_Year1:Depreciation_Year5)"

4. Suggestion Feedback

• General improvement suggestions
• Example: "Group similar expense categories together"

UDF Generation Process

1. Feedback Analysis

def _generate_udf_from_feedback(self, feedback_text, feedback_type, iteration):
    # Analyze feedback content and type
    # Generate appropriate UDF structure
    # Include metadata and version info

2. UDF Structure

def apply_user_feedback_v{iteration}(notes_data, feedback_type='{feedback_type}'):
    '''
    UDF generated from user feedback iteration {iteration}
    Feedback: {feedback_text}
    Type: {feedback_type}
    Generated: {timestamp}
    '''
    # Apply feedback-based modifications
    return notes_data

3. UDF Archiving

• All UDFs stored in session
• Versioned by iteration number
• Final UDF set upon approval

Session Management

Session States

• active: Accepting feedback and generating
• approved: Finalized with approved UDF
• cancelled: Terminated without approval

Session Data

{
    "session_id": "uuid-123",
    "status": "active",
    "current_iteration": 2,
    "feedback_history": [...],
    "archived_udfs": [...],
    "final_udf": null
}

Usage Workflow

Step 1: Initial Generation

curl -X POST http://localhost:8000/notes-llm \
  -F "file=@trial_balance.xlsx" \
  -o initial_notes.xlsx

Response Headers:

X-Session-ID: abc-123-def-456
X-Interactive-Enabled: true

Step 2: Submit Feedback

curl -X POST http://localhost:8000/notes-llm/feedback \
  -d "session_id=abc-123-def-456" \
  -d "feedback_text=Add more depreciation details" \
  -d "feedback_type=suggestion"

Step 3: Generate with Feedback

curl -X POST http://localhost:8000/notes-llm/generate \
  -F "session_id=abc-123-def-456" \
  -F "file=@updated_trial_balance.xlsx" \
  -o improved_notes.xlsx

Step 4: Approve Final Result

curl -X POST http://localhost:8000/notes-llm/approve \
  -d "session_id=abc-123-def-456"

Benefits

1. User-Driven Improvement

• Direct user feedback integration
• Iterative refinement based on actual needs

2. Learning System

• UDF functions capture user preferences
• Continuous improvement over time

3. Traceability

• Complete audit trail of changes
• Versioned feedback and UDFs

4. Flexibility

• Multiple feedback types supported
• Extensible UDF generation system

File Structure

data/
├── interactive_sessions.json    # Session storage
├── input/                       # Uploaded files
├── output/                      # Generated notes
└── udfs/                        # Archived UDF functions

Error Handling

Common Error Scenarios

• Invalid session ID
• Session already approved
• Feedback submission failures
• UDF generation errors

Error Responses

{
  "detail": "Session not found",
  "status_code": 404
}

Testing

Run the comprehensive test:

python test_interactive_feedback.py

This will test the complete workflow from initial generation through approval.

Future Enhancements

1. Advanced UDF Generation: Use LLM to create more sophisticated UDFs

2. Feedback Analytics: Analyze patterns in user feedback

3. Batch Processing: Apply approved UDFs to multiple files

4. Feedback Templates: Predefined feedback categories

5. Collaborative Sessions: Multiple users on same session c:\SAHIL\FinRyver\INTERACTIVE_FEEDBACK_README.md

6. User uploads trial balance → POST /notes-llm ↓

7. Initial notes generated + Session created ↓

8. User provides feedback → POST /notes-llm/feedback ↓

9. UDF generated and archived ↓

10. Generate improved notes → POST /notes-llm/generate ↓

11. User reviews and provides more feedback (loop) ↓

12. User approves final result → POST /notes-llm/approve ↓

13. Final UDF set and session completed