memops-small: Qwen3-0.6B LoRA memory-relation checker (experimental)

A LoRA adapter for Qwen/Qwen3-0.6B (revision c1899de289a04d12100db370d81485cdf75e47ca) that classifies the relation between a new candidate fact and one existing memory as strict JSON {"verdict": "same" | "conflict" | "unrelated"}. It was built during The Recursive Agents Hackathon (2026-10-17) for BrainBob, issue #7.

Experimental and opt-in. It names a semantic relation only. In BrainBob, deterministic rules (authority, scope, chronology, locks) decide what happens to memory; the model cannot delete anything, and malformed output falls back to the conservative path. It is not BrainBob's default judge.

Results (public synthetic diagnostic, not a hidden test)

Scored with the Bob benchmark scorer (codex/bob-benchmark@ab084a6) on 60 frozen synthetic pairs (20 per label):

Base + this adapter
Accuracy 32/60 (53.3%) 42/60 (70.0%)
Macro-F1 0.437 0.694
Invalid outputs 0 0

Accuracy delta +16.7 pp, template-cluster bootstrap 95% [+5.0, +28.3]. The tuned model labels 5 non-conflict pairs as conflict (base: 0), so it raises overwrite risk where an authorized correction is involved; the predictions and per-label probabilities are in runs/20261017-210457/.

Training

  • 144 synthetic training pairs (same 50 / conflict 44 / unrelated 50) generated by nvidia/NVIDIA-Nemotron-3-Super-120B-A12B, split by story template, 23 teacher labels corrected by hand. No company data.
  • LoRA r 16, alpha 32, dropout 0.05 on q/k/v/o; completion-only loss; non-thinking mode; lr 2e-4; 1 epoch; seed 42; bf16 on one NVIDIA L4 (9 steps, 8.4 s).
  • Script: code/job_all.py. Versions and full configuration: runs/20261017-210457/run_meta.json.

Limitations

  • Evaluated on 60 single-annotator synthetic pairs only; no independent label review and no comparison with a large judge model on the same set yet.
  • Half of real conflicts are still predicted unrelated.
  • English only; short single-fact pairs only.
Downloads last month
46
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Guangze/memops-small-qwen3-0.6b-lora

Finetuned
Qwen/Qwen3-0.6B
Adapter
(690)
this model