| title: README | |
| emoji: 🤟 | |
| colorFrom: green | |
| colorTo: gray | |
| sdk: static | |
| pinned: false | |
| # CrocoSign | |
| Machine translation from Polish Sign Language (PJM) video to text: an open dataset and baseline models, built as | |
| our engineering thesis at the Polish-Japanese Academy of Information Technology (Wojciech Sieńko, Rostyslav | |
| Kovalenko; supervisor Albert Śledzianowski). We no longer develop it, but everything stays available so that | |
| others can continue. Overview and results: [crocosign.pl](https://crocosign.pl); code: | |
| [github.com/crocosign](https://github.com/crocosign). | |
| - [pjm-data](https://huggingface.co/datasets/CrocoSign/pjm-data): 29,928 PJM video-sentence pairs (about 25 hours, 151 Deaf signers) with Polish transcripts, English translations, a multi-signer and a signer-independent split, features and poses. Built from the [Open Repository of the PJM Corpus](https://www.korpuspjm.uw.edu.pl/), University of Warsaw. | |
| - [pjm-segments](https://huggingface.co/datasets/CrocoSign/pjm-segments): all 39,817 sentence clips before filtering, as WebDataset shards, with a byte-offset index. | |
| - [pjm-models](https://huggingface.co/CrocoSign/pjm-models): Uni-Sign, SpaMo and our modified SpaMo, as ONNX and PyTorch checkpoints, with test outputs. The best single models reach 8.6 BLEU-4 on signers seen in training and 6.1 on unseen signers. | |
| The models are research prototypes, not a replacement for a sign language interpreter. Data: CC BY-SA 4.0, model weights: CC BY-NC-SA 4.0. | |