TROUBLESHOOTING_LLM_TIMEOUT.md

# Troubleshooting: LLM Timeout & Node.js Server Crashes

## Problem: App Hangs During Summarization / Node.js Server Stops

### Symptoms
- ✗ Application stops responding during "summarizing" phase
- ✗ Node.js server process terminates
- ✗ No error message, just hangs indefinitely
- ✗ Model loading takes forever or never completes

---

## ✅ IMMEDIATE FIX (Already Applied)

The enhanced version now includes:

1. **Aggressive Timeout Protection** (`llm_robust.py`)
   - Hard 60-second timeout (down from 120s)
   - Automatic fallback to lightweight processing
   - Emergency text-based analysis if LLM fails

2. **Optimized Configuration** (`.env` file created)
   - Lighter model recommendation (Mistral-7B vs Mixtral-8x7B)
   - Reduced token requirements (200 vs 300)
   - Faster failure detection

3. **Startup Health Check** (`start.sh` script)
   - Tests LLM connectivity before processing
   - Warns about configuration issues
   - Prevents hanging before it starts

---

## 🚀 Quick Start (Using Fixed Version)

### Option 1: Use Startup Script (Recommended)
```bash
cd /home/john/TranscriptorEnhanced

# Edit .env and add your HuggingFace token
nano .env

# Start with health check
./start.sh
```

### Option 2: Manual Start with Health Check
```bash
cd /home/john/TranscriptorEnhanced

# Test connectivity first
python3 fix_llm_timeout.py --test

# If test passes, start app
source .env
python3 app.py
```

---

## 🔧 Configuration Options

### .env File (Already Created)

```bash
# Option A: Use HuggingFace API (Most Stable - RECOMMENDED)
LLM_BACKEND=hf_api
HUGGINGFACE_TOKEN=your_token_here  # ← ADD YOUR TOKEN HERE
HF_MODEL=mistralai/Mistral-7B-Instruct-v0.2  # Lighter model

# Option B: Use LMStudio (Local - if you have it running)
LLM_BACKEND=lmstudio
LM_STUDIO_URL=http://localhost:1234

# Timeout Settings (Prevents Hanging)
LLM_TIMEOUT=60  # Hard timeout at 60 seconds
MAX_TOKENS_PER_REQUEST=200  # Reduced for speed
```

---

## 📋 Diagnostics

### Run Full Diagnostic
```bash
cd /home/john/TranscriptorEnhanced
python3 fix_llm_timeout.py --diagnose
```

### Test LLM Connectivity
```bash
python3 fix_llm_timeout.py --test
```

### Check Current Configuration
```bash
python3 fix_llm_timeout.py --config
```

---

## 🔍 Root Cause Analysis

### Why It Hangs

**1. Large Model + Limited Memory**
- Mixtral-8x7B requires ~30GB RAM
- Loading model exhausts memory
- Node.js/Python process killed by OS

**2. Network Timeouts**
- HuggingFace API unreachable
- Slow network connection
- Rate limiting

**3. Server Overload**
- Multiple concurrent requests
- LMStudio running out of resources
- GPU memory exhaustion

---

## ✅ Solutions Applied

### 1. Timeout Protection (`llm_robust.py`)

**Before:**
```python
# Waits indefinitely if model hangs
summary = query_llm(prompt, ...)
```

**After:**
```python
# Times out after 60s, uses fallback
with timeout(60):
    summary = query_llm(prompt, ...)
# Falls back to lightweight text extraction if timeout
```

### 2. Lightweight Fallbacks

If LLM times out, the system now:
1. Extracts data from the prompt text itself
2. Generates a lightweight summary with preserved data
3. Continues processing instead of crashing
4. Creates a report noting the limitation

**Example Fallback Output:**
```
LIGHTWEIGHT SUMMARY REPORT
(Generated due to LLM timeout - data extracted from available information)

SAMPLE OVERVIEW:
Total Patient interviews analyzed: 12

KEY OBSERVATIONS:
This analysis is based on structured data extraction rather than full LLM synthesis.

DATA EXTRACTED:
- Structured data preserved in CSV
- Individual transcript analyses completed
- Quantitative data available

RECOMMENDATIONS:
1. Reduce batch size (process fewer transcripts at once)
2. Verify LLM server connectivity
3. Consider lighter model (Mistral-7B vs Mixtral-8x7B)
```

### 3. Progressive Timeout Strategy

```
┌──────────────────────────────────────┐
│ Attempt 1: Full LLM (60s timeout)   │
└──────────┬───────────────────────────┘
           │
           ├─ Success → Continue normally
           │
           └─ Timeout → Fallback
                        ↓
           ┌──────────────────────────────────────┐
           │ Attempt 2: Lightweight extraction    │
           │ (Pattern-based, no LLM)              │
           └──────────┬───────────────────────────┘
                      │
                      ├─ Success → Continue with warning
                      │
                      └─ Failure → Emergency fallback
                                  ↓
                      ┌──────────────────────────────────────┐
                      │ Emergency: Preserve data only        │
                      │ (CSV export, minimal summary)        │
                      └──────────────────────────────────────┘
```

---

## 🎯 Recommended Settings by Use Case

### Small Datasets (1-5 transcripts)
```bash
LLM_BACKEND=hf_api
HF_MODEL=mistralai/Mistral-7B-Instruct-v0.2
LLM_TIMEOUT=90
MAX_TOKENS_PER_REQUEST=300
```

### Medium Datasets (6-15 transcripts)
```bash
LLM_BACKEND=hf_api
HF_MODEL=mistralai/Mistral-7B-Instruct-v0.2
LLM_TIMEOUT=60
MAX_TOKENS_PER_REQUEST=200
```

### Large Datasets (15+ transcripts) - Process in Batches
```bash
LLM_BACKEND=hf_api
HF_MODEL=mistralai/Mistral-7B-Instruct-v0.2
LLM_TIMEOUT=45
MAX_TOKENS_PER_REQUEST=150

# Process in batches of 10 transcripts max
```

---

## 🛠️ Manual Fixes

### If HuggingFace API is slow/timing out

**1. Get a HuggingFace Token**
```bash
# Visit: https://huggingface.co/settings/tokens
# Create a token
# Add to .env:
HUGGINGFACE_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxx
```

**2. Use Lighter Model**
```bash
# Edit .env:
HF_MODEL=mistralai/Mistral-7B-Instruct-v0.2  # Instead of Mixtral-8x7B
```

**3. Reduce Request Size**
```bash
# Edit .env:
MAX_TOKENS_PER_REQUEST=150
MAX_CHUNK_TOKENS=3000
```

### If Using LMStudio

**1. Start LMStudio Server**
```bash
# Open LMStudio
# Go to Server tab
# Start server on http://localhost:1234
```

**2. Load a Lightweight Model**
```bash
# In LMStudio, load one of:
- Mistral 7B Instruct
- Llama 2 7B Chat
- Phi-2

# Avoid heavy models:
- ✗ Mixtral 8x7B (too large)
- ✗ Llama 70B (too large)
```

**3. Configure .env**
```bash
LLM_BACKEND=lmstudio
LM_STUDIO_URL=http://localhost:1234
```

---

## 📊 Monitoring During Execution

The enhanced version now prints progress:

```
[Summary] Generating cross-transcript summary...
[Summary] Note: This may take 30-60 seconds for large datasets
[LLM] Starting summary generation...
[LLM] Timeout limit: 60s
[LLM] ✓ Completed successfully
[Summary] ✓ Validation passed (score: 0.85)
```

Watch for these messages:
- ✓ `Completed successfully` - All good
- ⚠ `Timeout after 60s` - Fallback activated
- ✗ `Using emergency fallback` - LLM completely unavailable

---

## 🔄 What Happens Now vs Before

### BEFORE (Hanging Behavior)

```
Processing transcripts... ✓
Extracting data... ✓
Generating summary...
[Waits indefinitely]
[Node.js crashes]
[No output]
```

### AFTER (Graceful Degradation)

```
Processing transcripts... ✓
Extracting data... ✓
Generating summary...
[LLM] Starting summary generation...
[LLM] Timeout limit: 60s
[LLM] ✗ Timeout after 60s
[LLM] Generating lightweight fallback...
[Summary] Using fallback summary
✓ Report generated with preserved data
```

---

## 📝 Testing the Fix

### Test 1: Verify Timeout Works
```bash
cd /home/john/TranscriptorEnhanced

# This should complete in <60s or fallback gracefully
python3 -c "
from llm_robust import query_llm_with_timeout
result = query_llm_with_timeout('Test', '', 'Other', max_timeout=10)
print('Success!' if result else 'Failed')
"
```

### Test 2: Full End-to-End
```bash
# Process a small transcript to verify
./start.sh
# Upload 1 transcript through UI
# Should complete in <2 minutes total
```

---

## 🚨 If Still Having Issues

### 1. Completely Bypass LLM (Emergency Mode)

Edit `/home/john/TranscriptorEnhanced/.env`:
```bash
# Force all LLM calls to use lightweight fallback
LLM_TIMEOUT=1  # 1 second timeout forces immediate fallback
```

This will:
- Skip LLM processing entirely
- Use pattern-based extraction only
- Generate reports from structured data
- Complete in seconds instead of minutes

### 2. Process One Transcript at a Time
Instead of batch processing, process individually through the UI.

### 3. Check System Resources
```bash
# Check available memory
free -h

# Check running processes
ps aux | grep -i "python\|node\|lmstudio"

# Kill stuck processes
pkill -f "python app.py"
pkill -f lmstudio
```

---

## ✅ Summary of Fixes

| Issue | Fix Applied | File |
|-------|-------------|------|
| Indefinite hangs | 60s hard timeout | `llm_robust.py` |
| No fallback | Lightweight text extraction | `llm_robust.py` |
| Server crashes | Graceful degradation | `app.py` |
| Heavy models | Lighter model recommendation | `.env` |
| No health check | Startup connectivity test | `fix_llm_timeout.py`, `start.sh` |

---

## 📞 Support

If issues persist:

1. **Check logs**: Console output shows exactly where it's failing
2. **Run diagnostic**: `python3 fix_llm_timeout.py --diagnose`
3. **Try emergency mode**: Set `LLM_TIMEOUT=1` in `.env`
4. **Process smaller batches**: 1-5 transcripts at a time

**The system will now always complete**, even if it has to fall back to lightweight processing. You'll get a report with preserved data regardless of LLM availability.

---

**Status:** ✅ Fixes Applied and Ready to Test
**Next Step:** Run `./start.sh` to start with health check