croco-corp/pjm-models

PJM Sign-Language → English — ONNX models

Repo-free ONNX exports of two PJM (Polish Sign Language) → English translation model families, served by the signsdk runtime (onnxruntime + bundled tokenizers/assets — no model-repo needed). All graphs fp32. Each model = projector.onnx + encoder.onnx + decoder.onnx (+ .onnx.data sidecar for SpaMo's flan-t5-xl) + meta.json + prompt/prefix embedding.

• Uni-Sign — pose-based (RTMPose keypoints) → mT5. Self-contained graphs.
• SpaMo / Mod-SpaMo — CLIP ViT-L/14 + VideoMAE-large (+ hand-CLIP + MediaPipe for mod-) → flan-t5-xl. Shares spamo/shared/ (CLIP+VideoMAE+hand-CLIP) across all SpaMo checkpoints.

Layout

assets/            mt5_tokenizer/ flan_t5_tokenizer/ yolo/yolo11n.onnx mediapipe/*.task
unisign/<name>/    projector.onnx encoder.onnx decoder.onnx prefix.npy meta.json
spamo/shared/      clip_vit_s2.onnx(+.data) videomae.onnx(+.data) hand_clip.onnx(+.data)
spamo/<name>/      projector.onnx(+.data) encoder.onnx(+.data) decoder.onnx(+.data) prompt_embeds.npy meta.json

Models (test-set BLEU-4, ✓ = uploaded)

model	family	source pretrain	split	BLEU-4	status
unisign-pjm-cslnews-ms	Uni-Sign	CSL-News	ms	2.42	✓
unisign-pjm-cslnews-si	Uni-Sign	CSL-News	si	1.43	✓
unisign-pjm-how2sign-ms	Uni-Sign	How2Sign	ms	7.61	✓
unisign-pjm-how2sign-si	Uni-Sign	How2Sign	si	4.77	—
unisign-pjm-openasl-ms	Uni-Sign	OpenASL	ms	7.38	—
unisign-pjm-openasl-si	Uni-Sign	OpenASL	si	4.79	—
unisign-pjm-scratch-ms	Uni-Sign	scratch	ms	1.62	—
unisign-pjm-scratch-si	Uni-Sign	scratch	si	0.72	—
spamo-pjm-phoenix-ms	SpaMo	PHOENIX	ms	5.54	✓
spamo-pjm-phoenix-si	SpaMo	PHOENIX	si	0.53	—
spamo-pjm-scratch-ms	SpaMo	scratch	ms	2.03	—
spamo-pjm-scratch-si	SpaMo	scratch	si	0.93	—
mod-spamo-pjm-phoenix-ms	Mod-SpaMo	PHOENIX	ms	5.06	✓
mod-spamo-pjm-phoenix-si	Mod-SpaMo	PHOENIX	si	0.47	—
mod-spamo-pjm-scratch-ms	Mod-SpaMo	scratch	ms	1.56	—
mod-spamo-pjm-scratch-si	Mod-SpaMo	scratch	si	1.17	—

BLEU-4 = reported PJM test score. Decoding faithful to the original eval (Uni-Sign beam=4; SpaMo beam-sample num_beams=5, top_p=0.9). fp32 chosen for deploy (fp16 overflows the bf16-trained flan-t5-xl; int8 is size-only).