CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models
Paper • 2305.14214 • Published
Quick Links
CompoundPiece model trained only on Stage 1 training data (self-supervised training on hyphenated and non-hyphenated words scraped from the web). See CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models.
Citation
@article{minixhofer2023compoundpiece,
title={CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models},
author={Minixhofer, Benjamin and Pfeiffer, Jonas and Vuli{\'c}, Ivan},
journal={arXiv preprint arXiv:2305.14214},
year={2023}
}
License
MIT
- Downloads last month
- 10
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
# Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("benjamin/compoundpiece-stage1") model = AutoModelForSeq2SeqLM.from_pretrained("benjamin/compoundpiece-stage1")