Lowca commited on
Commit
59cb2d6
·
verified ·
1 Parent(s): a500862

Document the SpaMo training checkpoints

Browse files
Files changed (1) hide show
  1. README.md +10 -7
README.md CHANGED
@@ -15,7 +15,7 @@ assets/ mt5_tokenizer/ flan_t5_tokenizer/ yolo/yolo11n.onnx mediapipe
15
  unisign/<name>/ projector.onnx encoder.onnx decoder.onnx prefix.npy meta.json
16
  spamo/shared/ clip_vit_s2.onnx(+.data) videomae.onnx(+.data) hand_clip.onnx(+.data)
17
  spamo/<name>/ projector.onnx(+.data) encoder.onnx(+.data) decoder.onnx(+.data) prompt_embeds.npy meta.json
18
- checkpoints/spamo/ spamo_pjm_ms_dual_bleu5.36.ckpt (PyTorch Lightning, not used by signsdk)
19
  ```
20
 
21
  ## Models (test-set BLEU-4, ✓ = uploaded)
@@ -43,10 +43,13 @@ BLEU-4 = reported PJM test score. Decoding faithful to the original eval
43
  (Uni-Sign beam=4; SpaMo beam-sample num_beams=5, top_p=0.9). fp32 chosen for deploy
44
  (fp16 overflows the bf16-trained flan-t5-xl; int8 is size-only).
45
 
46
- ## Training checkpoint
47
 
48
- `checkpoints/spamo/spamo_pjm_ms_dual_bleu5.36.ckpt` is a PyTorch Lightning checkpoint of SpaMo (PHOENIX-2014T init,
49
- PJM MS; epoch 43, dev BLEU-4 5.36). Its test run gave the 5.54 listed for `spamo-pjm-phoenix-ms`; which checkpoint
50
- the ONNX export was made from is not recorded.
51
- It is the starting point of the SpaMo-PJM `from_upstream` configs (v26 to v29). Moved here from the former
52
- `spamo-pjm-checkpoints` repo.
 
 
 
 
15
  unisign/<name>/ projector.onnx encoder.onnx decoder.onnx prefix.npy meta.json
16
  spamo/shared/ clip_vit_s2.onnx(+.data) videomae.onnx(+.data) hand_clip.onnx(+.data)
17
  spamo/<name>/ projector.onnx(+.data) encoder.onnx(+.data) decoder.onnx(+.data) prompt_embeds.npy meta.json
18
+ checkpoints/spamo/ spamo_pjm_ms_dual_bleu5.36.ckpt, <model>/*.ckpt (PyTorch Lightning, not used by signsdk)
19
  ```
20
 
21
  ## Models (test-set BLEU-4, ✓ = uploaded)
 
43
  (Uni-Sign beam=4; SpaMo beam-sample num_beams=5, top_p=0.9). fp32 chosen for deploy
44
  (fp16 overflows the bf16-trained flan-t5-xl; int8 is size-only).
45
 
46
+ ## Training checkpoints
47
 
48
+ PyTorch Lightning checkpoints of the SpaMo and Mod-SpaMo models above (not used by signsdk), for further training.
49
+ `checkpoints/spamo/<model>/` holds the checkpoint behind each table row; the BLEU-4 in a file name is the
50
+ validation score at that epoch, not the test score.
51
+
52
+ `checkpoints/spamo/spamo_pjm_ms_dual_bleu5.36.ckpt` is a SpaMo checkpoint (PHOENIX-2014T init, PJM MS; epoch 43,
53
+ dev BLEU-4 5.36). Its test run gave the 5.54 listed for `spamo-pjm-phoenix-ms`; which checkpoint the ONNX export
54
+ was made from is not recorded. It is the starting point of the SpaMo-PJM `from_upstream` configs (v26 to v29),
55
+ including the PHOENIX SI models. Moved here from the former `spamo-pjm-checkpoints` repo.