CLEF dual-task proper-protocol FT (seed 20261101, epoch 3) from jev-retrain MI355X run 2026-11-04

073de96 verified 2 days ago

1.94 kB

---
language:
- en
license: other
tags:
- security
- prompt-injection
- litert
---
# CLEF-DualScreen

Dual-task redo fine-tune of the CLEF-Flash JointSchemaHead
(121,762,820 parameters, 122 tensors) on frozen Cloudflare
CLEF-Flash LiteRT backbone hidden states. Only the head is trained.
Repository root is a complete CLEF LiteRT checkpoint directory.

## Training

- Compute: DigitalOcean droplet jev-retrain (AMD Instinct MI355X),
  finished 2026-11-04 21:36 UTC (training proper; see truncation note).
- Method: ft_clef.py original small AgentDojo protocol, 3 epochs,
  batch 16, lr 1e-4, pos_weight 3.0, seed 20261101.
- Data: train 569 / dev 149 / test 220 rows (workspace+travel
  train/dev, banking+slack test), both tasks mixed.
- Selected checkpoint: epoch 3 (saved_best by combined dev AUC).
- Full log: redo-training-log.jsonl. Hashes: redo-provenance.json.
## Evaluation (read before citing)
Dev is tiny (filter 65 rows, guard 84 rows) — treat all numbers as
noisy development measurements.
| split | tool_output_filter AUC | action_authorization_guard AUC |
|---|---|---|
| Dev epoch 0 (untrained) | 0.9781 | 0.9704 |
| Dev epoch 3 (training log) | 0.9829 | 0.9846 |
| Test 220 rows (independent local LiteRT rescore) | 1.0000 | 0.7731 |
Guard test detail (local rescore): TPR/FPR @0.35 = 0.983/0.600,
@0.5 = 0.967/0.517, @0.9 = 0.833/0.367. The guard task collapses
across domains (dev FPR@0.5 0.089 vs test 0.517) while the filter
task separates perfectly on this small test. Score statistic matches
training: filter P("true"), guard P("2") + 0.5*P("1"). Per-row test
scores: redo-test-scores-local.jsonl.
Truncation note: the droplet process was killed by SSH session-scope
deactivation during its own test-eval phase, after epoch-3 training
and checkpoint save completed. Test numbers above are therefore an
independent local rescore of the saved head, not the droplet log.
No frozen-chain replication exists yet.

	---
	language:
	- en
	license: other
	tags:
	- security
	- prompt-injection
	- litert
	---

	# CLEF-DualScreen

	Dual-task redo fine-tune of the CLEF-Flash JointSchemaHead
	(121,762,820 parameters, 122 tensors) on frozen Cloudflare
	CLEF-Flash LiteRT backbone hidden states. Only the head is trained.
	Repository root is a complete CLEF LiteRT checkpoint directory.

	## Training

	- Compute: DigitalOcean droplet jev-retrain (AMD Instinct MI355X),
	finished 2026-11-04 21:36 UTC (training proper; see truncation note).
	- Method: ft_clef.py original small AgentDojo protocol, 3 epochs,
	batch 16, lr 1e-4, pos_weight 3.0, seed 20261101.
	- Data: train 569 / dev 149 / test 220 rows (workspace+travel
	train/dev, banking+slack test), both tasks mixed.
	- Selected checkpoint: epoch 3 (saved_best by combined dev AUC).
	- Full log: redo-training-log.jsonl. Hashes: redo-provenance.json.

	## Evaluation (read before citing)

	Dev is tiny (filter 65 rows, guard 84 rows) — treat all numbers as
	noisy development measurements.

	\| split \| tool_output_filter AUC \| action_authorization_guard AUC \|
	\|---\|---\|---\|
	\| Dev epoch 0 (untrained) \| 0.9781 \| 0.9704 \|
	\| Dev epoch 3 (training log) \| 0.9829 \| 0.9846 \|
	\| Test 220 rows (independent local LiteRT rescore) \| 1.0000 \| 0.7731 \|

	Guard test detail (local rescore): TPR/FPR @0.35 = 0.983/0.600,
	@0.5 = 0.967/0.517, @0.9 = 0.833/0.367. The guard task collapses
	across domains (dev FPR@0.5 0.089 vs test 0.517) while the filter
	task separates perfectly on this small test. Score statistic matches
	training: filter P("true"), guard P("2") + 0.5*P("1"). Per-row test
	scores: redo-test-scores-local.jsonl.

	Truncation note: the droplet process was killed by SSH session-scope
	deactivation during its own test-eval phase, after epoch-3 training
	and checkpoint save completed. Test numbers above are therefore an
	independent local rescore of the saved head, not the droplet log.
	No frozen-chain replication exists yet.