CONTEXT.md

# Project Context: High-Throughput Diamond & Jewelry Vision Pipeline

## Domain
- **Application**: Industrial diamond processing (100,000+ daily videos)
- **Problem**: Segmentation, masking, and tracking of refractive/transparent gemstone objects
- **Input**: Sorted JPEG frame sequences (N-frame sequences from video capture)
- **Output**: Temporally-consistent soft-edge masks for downstream QC pipelines

## Core Stack & Models
- **Propagation Engine**: Meta SAM 3 — `VideoPredictor` API for multi-frame mask propagation + open-vocabulary text prompts
- **Student Detector**: YOLOv11-seg — Teacher-Student distillation for real-time inference
- **Temporal Smoothing**: `scipy.signal.savgol_filter` (Savitzky-Golay) applied per-frame mask coefficients
- **Performance Layer**: TensorRT export + batch inference (target: ≤10ms/frame)

## Refraction Rules (Strict)
1. **Never use binary masks** for diamonds or gemstones — always use soft-edge / alpha matting masks
2. **Alpha channel preservation**: Output masks must retain transparency gradients (float32 alpha map, 0.0–1.0)
3. **Edge softness**: Apply Gaussian-weighted alpha blending at mask boundaries (sigma ≥ 2px)
4. **Background reconstruction**: Use inpainting (e.g., `cv2.inpaint`) to handle semi-transparent regions

## Temporal Consistency Rules
1. **Savitzky-Golay filtering** MUST be applied across frame mask sequences (window=5, polyorder=2)
2. **No hard jumps**: Mask IoU between consecutive frames must be ≥ 0.85 (flag frames below threshold)
3. **Propagation priority**: Prefer SAM 3 propagation over per-frame YOLO prediction for tracked sequences
4. **Anchor frames**: Every 15th frame is re-annotated as a keyframe to prevent drift

## Architecture Rules
1. TensorRT FP16 for all inference — no FP32 in production paths
2. Batch size ≥ 8 for YOLO inference; ≥ 4 for SAM 3 propagation
3. Frame loading via OpenCV (`cv2.VideoCapture`) in sorted JPEG mode
4. All preprocessing must be GPU-side (CUDA streams)
5. Use `torch.compile()` for Python-side model wrappers where supported

## Naming Conventions
- Mask output files: `frame_{idx:06d}_mask_alpha.png`
- Model checkpoints: `yolo11seg_diamond_v{version}.pt` / `sam3_diamond_v{version}.pth`
- Dataset splits: `train/`, `val/`, `test/` under `data/`

## Quality Criteria
- Precision / Recall for diamond facets: target ≥ 0.92 mAP@0.5
- Mask temporal SSIM: ≥ 0.95 across N-frame sequences
- Throughput: ≥ 500 frames/sec on A100 80GB (batch mode)