request_lifecycle.md

# QualEdge: Query Lifecycle & Silicon Execution Architecture

This document describes the step-by-step lifecycle of a user query through the QualEdge hybrid edge-cloud router down to the processor silicon (Hexagon NPU vs. Kryo CPU), showing what optimizations solve quantization errors and what telemetry metrics are reported.

---

## 🗺️ Execution Sequence Flow

```mermaid
sequenceDiagram
    autonumber
    actor User as User / App
    participant Router as Hybrid Router (FastAPI)
    participant Classifier as TF-IDF & Logistic Regression
    participant LocalEngine as Local Inference (Qwen-0.5B)
    participant NPU as Hexagon NPU (HTP)
    participant CPU as Kryo CPU (Fallback)
    participant Verifier as Self-Verification Checker
    participant Cloud as Cloud Engine (Claude/Gemini)

    User->>Router: POST /api/router/route (Query prompt)
    
    rect rgb(15, 23, 42)
        Note over Router, Classifier: Pre-Processing & Routing Gate
        Router->>Classifier: Vectorize query prompt
        Classifier->>Classifier: Compute TF-IDF weights
        Classifier->>Classifier: Calculate complexity probability P(complex)
        Classifier-->>Router: Return P(complex) & pathway decision
    end

    alt P(complex) <= Threshold (Route On-Device)
        rect rgb(12, 74, 96)
            Note over Router, LocalEngine: Local Edge Inference
            Router->>LocalEngine: Ingest prompt & initialize model graph
            
            Note over LocalEngine, NPU: NPU Native Execution (Optimized with CLE & AdaRound)
            LocalEngine->>NPU: Offload INT8 Conv/GEMM operators (0.08 Joules)
            
            Note over LocalEngine, CPU: CPU Fallback Execution (Unmapped Operators)
            LocalEngine->>CPU: Run LayerNorm/Softmax on Kryo CPU (2.10 Joules penalty)
            
            LocalEngine-->>Router: Return generated text output
        end

        rect rgb(20, 83, 45)
            Note over Router, Verifier: Self-Verification check
            Router->>Verifier: Evaluate generated text for degradation
            Verifier->>Verifier: Calculate token repetition frequency
            
            alt Repetition < 40% & Not Null (Verification PASSED)
                Verifier-->>Router: Status: VERIFIED
                Router-->>User: Final Output + CLI Telemetry (NPU Energy, 0$ Cloud Cost)
            else Repetition >= 40% or Null (Verification FAILED - Local Model Collapse)
                Verifier-->>Router: Status: FAILED
                Note over Router, Cloud: Cloud Escalation Cascade (AutoMix)
                Router->>Cloud: Forward query to Claude/Gemini API
                Cloud-->>Router: Return high-quality cloud text
                Router-->>User: Final Output + CLI Telemetry (Escalated, Cloud Latency, $0.0055 Cost)
            end
        end

    else P(complex) > Threshold (Route Cloud Direct)
        rect rgb(15, 23, 42)
            Note over Router, Cloud: Direct Cloud Path
            Router->>Cloud: Forward query to Claude/Gemini API
            Cloud-->>Router: Return cloud response text
            Router-->>User: Final Output + CLI Telemetry (Direct Cloud, $0.0055 Cost)
        end
    end
```

---

## 🔍 Detailed Lifecycle Stages

### Stage 1: Ingestion & Feature Extraction (TF-IDF)
When the user submits a text query, the request hits the `/api/router/route` endpoint. Before passing the query to any neural network, it is pre-processed using a sub-1ms **TF-IDF Vectorizer**. This extracts semantic keyword features (e.g. keywords like `"scrapes"`, `"write"`, `"script"` indicate high reasoning complexity, whereas `"capital"`, `"what is"` indicate simple lookup tasks).

### Stage 2: Decision Routing Gate (Logistic Regression)
The extracted features are input to a **Logistic Regression classifier** that outputs the probability $P(\text{complex})$.
* If $P(\text{complex}) \le \theta$ (classification threshold, default `0.5`), the query is routed to the on-device path.
* If $P(\text{complex}) > \theta$, the query bypasses local execution and goes directly to the cloud, preventing local model failure on complex prompts.

### Stage 3: Edge NPU Execution & Operator Graph Allocation
When executing on-device, the model graph is partitioned:
1. **Hexagon Tensor Processor (HTP - NPU):** Handles the core weight-matrix multiplies. 
   * *What We Solved:* Naive INT4/INT8 quantization causes accuracy degradation. We applied **Cross-Layer Equalization (CLE)** to balance convolution weight channels and **AdaRound** to dynamically calculate optimal weight rounding values per layer, maintaining Top-1 accuracy.
   * *Silicon Draw:* Drew only **~0.08 Joules** of energy per query on native HTP silicon.
2. **Kryo CPU (Fallback):** General-purpose CPU handles layers that the NPU compiler lacks kernel maps for (e.g. specialized normalization, resizing, or custom activation layers).
   * *The Penalty:* Moving tensors back-and-forth from NPU to CPU cache triggers an energy penalty of **~2.10 Joules** per query.

### Stage 4: Output Self-Verification Check
Quantized edge models (like Qwen-0.5B) are prone to **quantization collapse** (loops of repetitive tokens, e.g. `"tokyo tokyo tokyo..."` or empty strings) when facing challenging inputs.
* *Our Solution:* A self-verification checker evaluates the generated output in real-time. If the token repetition rate exceeds **40%**, it rejects the local output.

### Stage 5: Cloud Escalation Cascade (AutoMix)
If the verification checker fails the local response, the router automatically spins up a cloud client fallback, sending the query to the **Cloud Inference Engine** (Gemini/Claude) and recovering response quality seamlessly for the user.

---

## 💻 CLI Telemetry Output (What You See)

When a query is processed, the system prints the exact execution trace and silicon metrics to the command line.

### Scenario A: Local NPU Execution (Success)
```bash
[ROUTER] Ingesting query: "What is the capital of Japan?"
[ROUTER] Pre-processing TF-IDF features... (Time: 0.15ms)
[ROUTER] Routing Decision: ON_DEVICE (Complexity Score: 0.12, Threshold: 0.50)
[ROUTER] Executing locally on Snapdragon X Elite NPU...
[ROUTER] HTP Silicon Allocation: 98% native HTP execution, 2% Kryo CPU fallback.
[ROUTER] Local generation complete. Checking output quality...
[ROUTER] Output quality self-verification: PASSED (Repetition index: 0.00)
[ROUTER] ================= TELEMETRY =================
[ROUTER] Latency: 87.2ms
[ROUTER] Energy Consumed: 0.12 Joules
[ROUTER] Cloud API Cost: $0.0000 USD
[ROUTER] =============================================
```

### Scenario B: Local Execution with Cloud Escalation (Self-Verification Fallback)
```bash
[ROUTER] Ingesting query: "Draft a polite email asking for feedback on my project."
[ROUTER] Pre-processing TF-IDF features... (Time: 0.18ms)
[ROUTER] Routing Decision: ON_DEVICE (Complexity Score: 0.44, Threshold: 0.50)
[ROUTER] Executing locally on Snapdragon X Elite NPU...
[ROUTER] Local generation complete. Checking output quality...
[ROUTER] WARNING: Output failed quality validation (Repetitive loop detected, score: 0.76).
[ROUTER] Escalating query to cloud fallback (AutoMix cascade)...
[ROUTER] Cloud model response received successfully.
[ROUTER] ================= TELEMETRY =================
[ROUTER] Latency: 896.5ms (86.5ms local + 810.0ms cloud)
[ROUTER] Energy Consumed: 2.22 Joules (0.12J local NPU + 2.10J CPU fallback penalty)
[ROUTER] Cloud API Cost: $0.0055 USD
[ROUTER] =============================================
```