Document the SpaMo training checkpoints
Browse files
README.md
CHANGED
|
@@ -15,7 +15,7 @@ assets/ mt5_tokenizer/ flan_t5_tokenizer/ yolo/yolo11n.onnx mediapipe
|
|
| 15 |
unisign/<name>/ projector.onnx encoder.onnx decoder.onnx prefix.npy meta.json
|
| 16 |
spamo/shared/ clip_vit_s2.onnx(+.data) videomae.onnx(+.data) hand_clip.onnx(+.data)
|
| 17 |
spamo/<name>/ projector.onnx(+.data) encoder.onnx(+.data) decoder.onnx(+.data) prompt_embeds.npy meta.json
|
| 18 |
-
checkpoints/spamo/ spamo_pjm_ms_dual_bleu5.36.ckpt (PyTorch Lightning, not used by signsdk)
|
| 19 |
```
|
| 20 |
|
| 21 |
## Models (test-set BLEU-4, ✓ = uploaded)
|
|
@@ -43,10 +43,13 @@ BLEU-4 = reported PJM test score. Decoding faithful to the original eval
|
|
| 43 |
(Uni-Sign beam=4; SpaMo beam-sample num_beams=5, top_p=0.9). fp32 chosen for deploy
|
| 44 |
(fp16 overflows the bf16-trained flan-t5-xl; int8 is size-only).
|
| 45 |
|
| 46 |
-
## Training
|
| 47 |
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
`
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
unisign/<name>/ projector.onnx encoder.onnx decoder.onnx prefix.npy meta.json
|
| 16 |
spamo/shared/ clip_vit_s2.onnx(+.data) videomae.onnx(+.data) hand_clip.onnx(+.data)
|
| 17 |
spamo/<name>/ projector.onnx(+.data) encoder.onnx(+.data) decoder.onnx(+.data) prompt_embeds.npy meta.json
|
| 18 |
+
checkpoints/spamo/ spamo_pjm_ms_dual_bleu5.36.ckpt, <model>/*.ckpt (PyTorch Lightning, not used by signsdk)
|
| 19 |
```
|
| 20 |
|
| 21 |
## Models (test-set BLEU-4, ✓ = uploaded)
|
|
|
|
| 43 |
(Uni-Sign beam=4; SpaMo beam-sample num_beams=5, top_p=0.9). fp32 chosen for deploy
|
| 44 |
(fp16 overflows the bf16-trained flan-t5-xl; int8 is size-only).
|
| 45 |
|
| 46 |
+
## Training checkpoints
|
| 47 |
|
| 48 |
+
PyTorch Lightning checkpoints of the SpaMo and Mod-SpaMo models above (not used by signsdk), for further training.
|
| 49 |
+
`checkpoints/spamo/<model>/` holds the checkpoint behind each table row; the BLEU-4 in a file name is the
|
| 50 |
+
validation score at that epoch, not the test score.
|
| 51 |
+
|
| 52 |
+
`checkpoints/spamo/spamo_pjm_ms_dual_bleu5.36.ckpt` is a SpaMo checkpoint (PHOENIX-2014T init, PJM MS; epoch 43,
|
| 53 |
+
dev BLEU-4 5.36). Its test run gave the 5.54 listed for `spamo-pjm-phoenix-ms`; which checkpoint the ONNX export
|
| 54 |
+
was made from is not recorded. It is the starting point of the SpaMo-PJM `from_upstream` configs (v26 to v29),
|
| 55 |
+
including the PHOENIX SI models. Moved here from the former `spamo-pjm-checkpoints` repo.
|