# High-Level Design (HLD)

## System Overview

Music Generation LLM is a symbolic music generation system that learns patterns from
classical MIDI music and generates new compositions. It operates entirely on symbolic
note representations (MIDI events), not raw audio waveforms.

## Architecture Diagram

```
┌─────────────────────────────────────────────────────────┐
│                    USER INTERFACE                        │
│              CLI: train / generate / both                │
└────────────────────────┬────────────────────────────────┘
                         │
┌────────────────────────▼────────────────────────────────┐
│                  ORCHESTRATOR (s00_main)                 │
│              Parses args, wires components               │
└──────┬─────────────────┬────────────────────┬───────────┘
       │                 │                    │
┌──────▼──────┐  ┌───────▼───────┐  ┌────────▼────────┐
│   DATA      │  │    MODEL      │  │   GENERATION    │
│  PIPELINE   │  │   PIPELINE    │  │   PIPELINE      │
├─────────────┤  ├───────────────┤  ├─────────────────┤
│ HF Download │  │ MusicTrans-   │  │ Autoregressive  │
│ MIDI Parse  │  │ former Build  │  │ w/ KV-cache     │
│ REMI Token  │  │ Train Loop    │  │ Top-p/k Sample  │
│ DataLoader  │  │ AMP + Ckpt    │  │ MIDI Export     │
└─────────────┘  └───────────────┘  └─────────────────┘
```

## Component Responsibilities

### Data Pipeline (s02 + s03)
- Download 4,796 MIDI files from HuggingFace
- Parse with `pretty_midi` library
- Tokenize using REMI scheme (Note On/Off, Velocity, TimeShift, Bar, Position)
- Create PyTorch DataLoaders with padding and random cropping

### Model (s04)
- LLaMA-style Transformer (6 layers, 256 dim, 8 heads)
- Grouped Query Attention with 4 KV heads (50% memory reduction)
- SwiGLU feed-forward with RMSNorm
- RoPE for positional encoding
- Gradient checkpointing support

### Training (s05)
- AdamW optimizer with cosine LR warmup
- Mixed precision (BF16/FP16) via PyTorch AMP
- Gradient accumulation (effective batch = 32)
- Early stopping with patience
- TensorBoard logging

### Generation (s06)
- Autoregressive decoding with KV-cache
- Temperature scaling + Top-k + Top-p (nucleus) sampling
- Repetition penalty for diverse output
- Direct MIDI file export

## Data Flow

```
MIDI Files → REMI Tokens → Training → Trained Model → Generation → MIDI Output
```

## Constraints & Decisions

| Decision                     | Rationale                                    |
|------------------------------|----------------------------------------------|
| Symbolic (MIDI) not audio    | 100x smaller data, trainable on consumer GPU |
| GQA over standard MHA        | 50% KV-cache memory reduction                |
| Gradient checkpointing       | ~50% memory savings, ~20% speed cost         |
| BF16 mixed precision         | 50% memory, maintains numeric stability      |
| REMI tokenization            | Best published results for symbolic music     |
| 5M params (not 100M+)        | Fits in <=4GB VRAM, fast training             |