docs/usecase.md

# CommitGuard — Use Cases & Test Scenarios

This document outlines the primary use cases and associated test scenarios for running CommitGuard as a standalone Command Line Interface (CLI) tool and as an integrated Plugin (e.g., CI/CD Pipeline or IDE Extension).

## 1. CommitGuard as a CLI (Standalone Workflow)
This use case is for security researchers, data scientists, and ML engineers training or evaluating the model locally or on a dedicated VM.

### 1.1 Data Preprocessing
- **Scenario:** Convert raw Devign JSON into a filtered, balanced, 5000-sample JSONL file.
- **Action:** Run `python scripts/preprocess_devign.py --limit 5000`
- **Expected Result:** `data/devign_filtered.jsonl` is created with clean, XML-ready code diffs and valid `cwe` labels.

### 1.2 Environment Server (OpenEnv)
- **Scenario:** Start the RLVR training environment.
- **Action:** Run `python -m commitguard_env.server`
- **Expected Result:** Server starts on port 8000. `curl http://localhost:8000/health` returns `{"status": "healthy"}`. `tests/test_no_leak.py` confirms no label leakage in `/reset` or `/state`.

### 1.3 Model Training (GRPO)
- **Scenario:** Train the Llama-3.2-3B model using the live RLVR environment.
- **Action:** Run `python scripts/train_grpo.py --live --steps 500`
- **Expected Result:** Model trains using 4-bit quantization and LoRA. Training curve uploads to WandB. Checkpoints save every 50 steps.

### 1.4 Agentic Evaluation
- **Scenario:** Evaluate the trained LoRA adapter on 100 held-out test samples.
- **Action:** Run `python scripts/evaluate.py --adapter_path ./outputs/commitguard-final`
- **Expected Result:** The agent executes a 5-step loop (request_context -> analyze -> verdict). A detailed `eval_results.json` report is generated showing accuracy per CWE.

### 1.5 Visualization
- **Scenario:** Generate performance plots for reporting.
- **Action:** Run `python plots/plot_baseline_vs_trained.py`
- **Expected Result:** A PNG bar chart is saved showing the clear accuracy delta between baseline and trained model.

---

## 2. CommitGuard as a Plugin (Developer Workflow)
This use case is for software engineers interacting with the trained model during their daily development cycle to prevent vulnerabilities from reaching production.

### 2.1 Git Pre-Commit Hook (Local Plugin)
- **Scenario:** A developer attempts to commit code containing an SQL injection (e.g., `CWE-89`).
- **Action:** Developer runs `git commit -m "Update user query"`. The hook captures the local diff and invokes the CommitGuard agent API.
- **Expected Result:**
  - The agent detects the vulnerability before the commit executes.
  - The commit is **blocked** (exit code 1).
  - The terminal outputs the agent's XML `exploit_sketch`: `"SQL injection in user_id via f-string construction."`

### 2.2 CI/CD Pull Request Reviewer (GitHub Action)
- **Scenario:** A developer opens a Pull Request with a new feature.
- **Action:** GitHub Actions triggers a CommitGuard workflow container. The agent runs a full evaluation loop over the PR's diff patch.
- **Expected Result:**
  - The agent posts an automated review comment directly on the PR.
  - If vulnerable, it flags the specific line and provides a remediation suggestion.
  - The PR status check turns **Red (Failed)** if a severe vulnerability is detected, preventing a merge to the main branch.

### 2.3 IDE Extension (VS Code / Cursor Integration)
- **Scenario:** Real-time vulnerability detection while typing.
- **Action:** Developer saves a file (`Ctrl+S`). The IDE plugin sends the local file diff to a hosted CommitGuard backend.
- **Expected Result:**
  - The agent identifies an issue using its `analyze` action step.
  - A diagnostic warning (red squiggly line) appears under the vulnerable code snippet in the editor.
  - Hovering shows the agent's `<reasoning>` and suggested safe implementation.