metadata
title: README
emoji: 🤟
colorFrom: green
colorTo: gray
sdk: static
pinned: false
CrocoSign
Machine translation from Polish Sign Language (PJM) video to text: an open dataset and baseline models, built as our engineering thesis at the Polish-Japanese Academy of Information Technology (Wojciech Sieńko, Rostyslav Kovalenko; supervisor Albert Śledzianowski). We no longer develop it, but everything stays available so that others can continue. Overview and results: crocosign.pl; code: github.com/crocosign.
- pjm-data: 29,928 PJM video-sentence pairs (about 25 hours, 151 Deaf signers) with Polish transcripts, English translations, a multi-signer and a signer-independent split, features and poses. Built from the Open Repository of the PJM Corpus, University of Warsaw.
- pjm-segments: all 39,817 sentence clips before filtering, as WebDataset shards, with a byte-offset index.
- pjm-models: Uni-Sign, SpaMo and our modified SpaMo, as ONNX and PyTorch checkpoints, with test outputs. The best single models reach 8.6 BLEU-4 on signers seen in training and 6.1 on unseen signers.
The models are research prototypes, not a replacement for a sign language interpreter. Data: CC BY-SA 4.0, model weights: CC BY-NC-SA 4.0.