Token Classification
Transformers
PyTorch
TensorFlow
ONNX
Safetensors
xlm-roberta
punctuation prediction
punctuation
Instructions to use oliverguhr/fullstop-punctuation-multilang-large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use oliverguhr/fullstop-punctuation-multilang-large with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="oliverguhr/fullstop-punctuation-multilang-large")# Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("oliverguhr/fullstop-punctuation-multilang-large") model = AutoModelForTokenClassification.from_pretrained("oliverguhr/fullstop-punctuation-multilang-large", device_map="auto") - Inference
- Notebooks
- Google Colab
- Kaggle
End of Sentence Punctuation Only
#3
by cooperchris17 - opened
Is it possible to only restore end of sentence punctuation (full stops), and not all of the other punctuation marks?
(Sorry if this is an obvious question, but I couldn't work out the answer from reading the code, as I'm still new to NLP)
This model always predicts the probabilities of punctuation marks - if you just need the the full stops than just use them and ignore the others.
If you want to segment sentences you can check if the label is a . or ? and assume its an sentence end.
Thank you, I understand now, I can use 'model.predict()' in your example code and choose the punctuation that I want.
This model is very useful, thank you for making it available!
oliverguhr changed discussion status to closed