README.md

---
base_model: Qwen/Qwen2.5-VL-7B-Instruct
library_name: peft
tags:
  - lora
  - qwen2_5_vl
  - video
  - streaming
  - step-completion
license: other
---

# astertech-cvd-coach-lora-r0

LoRA adapter for **Qwen/Qwen2.5-VL-7B-Instruct** that performs *streaming step-completion
detection* on egocentric video of CVD (chemical vapour deposition) tube-furnace
experiments. At each tick the model is shown the last 30 s of footage plus the current
instruction and answers a Yes/No question about whether that step is now complete, so a
coach can acknowledge the step and advance.

This is **round 0** of the AsterTech-vLLM project (milestone M3). Completion detection
only — mistake detection is deliberately out of scope.

## Provenance and access

Trained on **local-only laboratory footage** that is not distributed and never leaves the
project's own machines. The weights here are derived from that footage, which is why this
repository is **private**. Treat any change of visibility as a deliberate decision about
derived data, not a routine toggle.

## Training recipe

Reproduced verbatim from `results/m3/pooled_provenance.json` (seed 0, deterministic
inputs: hash-pinned annotations + frozen benchmark manifest `benchmarks/v1.json`).

| | |
|---|---|
| Base model | `Qwen/Qwen2.5-VL-7B-Instruct` (bf16) |
| LoRA | r=16, α=32, dropout=0.05 |
| Optimiser | lr 1e-5, batch 1 × grad-accum 8, 1 epoch, seed 0 |
| Frames | 2 fps, ≤16 frames/example, 30 s clip window, `max_pixels` 151200 |
| Pair generation | `completion_pairs_from_gt(short_yes=True)`, `neg_per_step=4`, `neg_margin_s=8`, `min_step_s=12` |
| Dataset | **672 examples — 448 Yes / 224 No** (56 neg-eligible steps ×5 + 392 yes-only + 3 zero-length skips) |
| Train videos | the 13 `pool` videos of benchmark v1 (the 3 frozen holdout videos are never trained on) |

Training data is **same-step contrastive pairs**: for each annotated step one "Yes" window
ending at the completion time and N "No" windows ending mid-step, with *identical
instruction text and window length*, so only the pixels differ. Every example comes from
one code path over the project's own ground truth — no external corpora, no mixed sources.

## Results (M3, benchmark v1)

Evaluated at the frozen v1 eval config (tick 5 s / window 30 s / 2 fps / ≤16 frames /
confirm 1; match window 30 s full width).

| split | zero-shot IC-Acc | SFT IC-Acc | delta | missed | overtalk |
|---|---|---|---|---|---|
| pool (LOVO, 13 folds) | 14.63% (66/451) | **48.78%** (220/451) | +34.15 | 0.854 → 0.512 | 0.283 → 0.315 |
| frozen holdout (this pooled adapter) | 20.48% (17/83) | **54.22%** (45/83) | +33.74 | 0.795 → 0.458 | 0.292 → 0.262 |

The holdout videos were never trained on in any run, and gained as much as the LOVO folds
— which is what rules out "it memorised the training videos".

Slice movement on the pool (zero-shot → SFT): short steps 8/80 → 49/80, pressure waits
1/64 → 18/64, other 57/311 → 154/311.

## About this copy — a reproduction, not the original artifact

The original M3 pooled adapter was written to instance-store disk and lost when the
training box was stopped. These weights are a **re-run of the identical recipe** on
2026-08-04: same committed annotations, same hash-pinned manifest, same 13 train ids,
`seed 0`, same hyperparameters. The reproduction is exact on every input we can check —
the dataset came out at 672 examples / 448 Yes / 224 No, matching the original pin
byte-for-byte, and training took 66.9 min against the original's 67.0 min, ending at
`train_loss` 0.4088.

GPU nondeterminism means the weights are not bit-identical to the originals, so the table
above is the *original* run's measurement. See "Verification" below for this copy's own
measured holdout score.

## Caveats — read these before trusting it

1. **Overtalk rose on the pool** (0.283 → 0.315). The 2:1 Yes bias buys detections partly
   through more emissions. Holdout overtalk *improved* (0.292 → 0.262), so this is not a
   degenerate always-yes model, but it is a trade. The next lever is harder mid-step
   negative windows.
2. **The pressure-wait gain is not gauge reading.** The DigiVac LCD is provably illegible
   at the model's 360×420 effective input, so the model is learning the operator's
   reaction cue (stillness, then reaching for the next valve), not the displayed value.
   Do not present it as instrument reading.
3. **Small dataset.** 16 videos / 534 completions total. Single-video scores (n = 21–49)
   are noise; only the aggregates above carry signal.

## Verification

These exact weights were re-scored on the 3 frozen holdout videos (2026-08-04), loaded
from this repository by id — so the number below also verifies the download path, not just
the local files.

| metric | original M3 adapter | this copy |
|---|---|---|
| IC-Acc | 54.22% (45/83) | **57.83%** (48/83) |
| missed | 0.458 | 0.422 |
| overtalk | 0.262 | 0.238 |
| mean signed offset | — | +1.02 s (abs 4.19 s) |

+3.61 pts against a ±5 pt reproduction gate — pass. The difference is GPU nondeterminism
at n=83 (3 completions), not a better model; both are the same recipe and the original
table is the one to quote. Zero-shot on the same 3 videos is 20.48% (17/83).

## Usage

```python
from peft import PeftModel
from transformers import AutoModelForVision2Seq

base = AutoModelForVision2Seq.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct", dtype="bfloat16")
model = PeftModel.from_pretrained(base, "Rnoooo/astertech-cvd-coach-lora-r0")
```

Requires authentication (`HF_TOKEN`) while the repository is private. The adapter expects
the exact Yes/No completion prompt shape it was trained on — the project's
`astertech.prompts.completion_messages`. Using a different prompt string collapses recall;
that was the central bug of an earlier round.