memory-bank/systemPatterns.md

# System Patterns

## Architecture Overview
```
┌─────────────────────────────────────────┐
│              FastAPI App                │
├─────────────────────────────────────────┤
│  Routes:                                │
│  • GET / (Welcome)                      │
│  • POST /download (Model Download)      │
│  • POST /v1/chat/completions (Chat)     │
├─────────────────────────────────────────┤
│  Global State:                          │
│  • pipe (Pipeline)                      │
│  • tokenizer (Tokenizer)                │
│  • model_name (Current Model)           │
├─────────────────────────────────────────┤
│  Startup Event:                         │
│  • Load .env                            │
│  • Initialize default model             │
└─────────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────┐
│           Utils Modules                 │
├─────────────────────────────────────────┤
│  utils/model.py:                        │
│  • check_model() - Verify model exists  │
│  • download_model() - Download model    │
│  • initialize_pipeline() - Setup model  │
│  • DownloadRequest - Pydantic model     │
├─────────────────────────────────────────┤
│  utils/chat_request.py:                 │
│  • ChatRequest - Request validation     │
├─────────────────────────────────────────┤
│  utils/chat_response.py:                │
│  • create_chat_response() - Generate    │
│  • convert_json_format() - Parse output │
│  • ChatResponse/ChatChoice/ChatUsage    │
└─────────────────────────────────────────┘
```

## Data Flow Patterns

### 1. Application Startup
```
.env → load_dotenv() → os.getenv("DEFAULT_MODEL_NAME")
     ↓
initialize_pipeline(model_name)
     ↓
check_model() → verify cache exists
     ↓
AutoTokenizer + AutoModelForCausalLM
     ↓
pipeline("text-generation")
     ↓
Global: pipe, tokenizer, model_name
```

### 2. Chat Request Flow
```
POST /v1/chat/completions
     ↓
ChatRequest (validation)
     ↓
Check model_name match
     ↓
create_chat_response(request, pipe, tokenizer)
     ↓
pipe(messages, max_new_tokens)
     ↓
convert_json_format() → clean output
     ↓
Calculate tokens (tokenizer.encode)
     ↓
ChatResponse (Pydantic)
```

### 3. Download Flow
```
POST /download
     ↓
download_model(model_name)
     ↓
AutoTokenizer.from_pretrained(cache_dir)
AutoModelForCausalLM.from_pretrained(cache_dir)
     ↓
initialize_pipeline(model_name)
     ↓
Update global: pipe, tokenizer, model_name
     ↓
Return success + loaded status
```

## Key Design Decisions

### 1. Global State Management
- **Why**: FastAPI is stateless, but models are expensive to load
- **Solution**: Global variables for pipe/tokenizer/model_name
- **Trade-off**: Single model at a time, but efficient

### 2. Lazy Initialization with Fallback
- **Why**: Model might not exist on startup
- **Solution**: Startup event tries to load, but doesn't fail
- **Trade-off**: Graceful degradation vs. guaranteed availability

### 3. Model Switching
- **Why**: Users may want different models
- **Solution**: Check request.model vs. current model_name
- **Trade-off**: Re-initialization overhead vs. flexibility

### 4. Error Handling
- **Why**: Model operations can fail in multiple ways
- **Solution**: HTTPException for client errors, try/except for internal
- **Trade-off**: Clear API vs. implementation complexity

### 5. Environment Configuration
- **Why**: Different deployments need different defaults
- **Solution**: .env file with fallback
- **Trade-off**: External config vs. hardcoded values

## Security Considerations
- ✅ No hardcoded credentials in code
- ✅ HUGGINGFACE_TOKEN from environment
- ✅ Input validation via Pydantic
- ✅ No arbitrary code execution from user input

## Performance Patterns
- ✅ Model loaded once at startup
- ✅ Tokenizer reused across requests
- ✅ Token counting with actual tokenizer
- ✅ Async route handlers for concurrency