File size: 1,681 Bytes
c7805b7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | # v1.0 Verification Results
## Summary
| Metric | Value | Threshold | Result |
|--------|-------|-----------|--------|
| retrieval_similarity | 0.7239 | >= 0.60 | PASS |
| tIOU | 0.6700 | >= 0.40 | PASS |
| mAP@0.5 | 0.5092 | >= 0.50 | PASS |
**Overall: PASS — v1.0 ACCEPTED**
## Details
- Checkpoint: `outputs/checkpoint-770` (step 770, eval_loss 0.5272)
- Predictions: `checkpoints/v21_semantic_boundaries/predictions.jsonl` (98 samples)
- Embedder: OpenAI text-embedding-3-small (production-aligned)
- Date: 2026-03-19
### Retrieval Similarity
- Mean: 0.7239
- Median: 0.7355
- Min: 0.4982, Max: 0.9162
- Excellent (>= 0.80): 22/98
- Good (>= 0.60): 87/98
- Acceptable (>= 0.40): 98/98
### Temporal IoU (tIOU)
- Mean: 0.6700
- Median: 0.6703
- Predictions with timestamps: 98/98
- References with timestamps: 98/98
### mAP@0.5
- Mean: 0.5092
## Production Embeddings (prior run)
For reference, the prior eval with OpenAI text-embedding-3-small on 20 pairs showed:
- mean_similarity: 0.6781
- production_ready: true
## Failure Analysis
All metrics pass. v1.0 is accepted for production.
## Per-Sample Scores (first 10)
| # | retrieval_sim | tIOU | mAP@0.5 | pred_events | ref_events |
|---|--------------|------|---------|-------------|------------|
| 0 | 0.6549 | 0.5277 | 0.4500 | 5 | 4 |
| 1 | 0.7739 | 0.8080 | 1.0000 | 5 | 5 |
| 2 | 0.6433 | 1.0000 | 1.0000 | 4 | 4 |
| 3 | 0.7473 | 0.6228 | 0.4500 | 5 | 4 |
| 4 | 0.7774 | 0.5406 | 0.2500 | 4 | 4 |
| 5 | 0.7924 | 0.3868 | 0.0500 | 4 | 5 |
| 6 | 0.7626 | 0.7779 | 0.7500 | 4 | 3 |
| 7 | 0.6476 | 0.8060 | 0.7500 | 4 | 3 |
| 8 | 0.7355 | 0.7520 | 0.6400 | 5 | 5 |
| 9 | 0.8859 | 0.6579 | 0.5625 | 4 | 4 |
|