File size: 1,488 Bytes
373a896 7246b92 373a896 7246b92 27001ac 7246b92 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | ---
title: README
emoji: 🤟
colorFrom: green
colorTo: gray
sdk: static
pinned: false
---
# CrocoSign
Machine translation from Polish Sign Language (PJM) video to text: an open dataset and baseline models, built as
our engineering thesis at the Polish-Japanese Academy of Information Technology (Wojciech Sieńko, Rostyslav
Kovalenko; supervisor Albert Śledzianowski). We no longer develop it, but everything stays available so that
others can continue. Overview and results: [crocosign.pl](https://crocosign.pl); code:
[github.com/crocosign](https://github.com/crocosign).
- [pjm-data](https://huggingface.co/datasets/CrocoSign/pjm-data): 29,928 PJM video-sentence pairs (about 25 hours, 151 Deaf signers) with Polish transcripts, English translations, a multi-signer and a signer-independent split, features and poses. Built from the [Open Repository of the PJM Corpus](https://www.korpuspjm.uw.edu.pl/), University of Warsaw.
- [pjm-segments](https://huggingface.co/datasets/CrocoSign/pjm-segments): all 39,817 sentence clips before filtering, as WebDataset shards, with a byte-offset index.
- [pjm-models](https://huggingface.co/CrocoSign/pjm-models): Uni-Sign, SpaMo and our modified SpaMo, as ONNX and PyTorch checkpoints, with test outputs. The best single models reach 8.6 BLEU-4 on signers seen in training and 6.1 on unseen signers.
The models are research prototypes, not a replacement for a sign language interpreter. Data: CC BY-SA 4.0, model weights: CC BY-NC-SA 4.0.
|