Token Classification
Transformers
Safetensors
Malagasy
xlm-roberta
punctuation-restoration
truecasing
malagasy
asr-postprocessing
Instructions to use Flo976/malagasy-punctuation with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Flo976/malagasy-punctuation with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="Flo976/malagasy-punctuation")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("Flo976/malagasy-punctuation") model = AutoModelForTokenClassification.from_pretrained("Flo976/malagasy-punctuation", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Download labels.json from Flo976/malagasy-punctuation: direct link, hf CLI and curl.
- Browser
- Download file 809 Bytes
-
https://huggingface.co/Flo976/malagasy-punctuation/resolve/main/labels.json
- Command line
-
hf download hf://Flo976/malagasy-punctuation/labels.json
-
curl -L -o labels.json https://huggingface.co/Flo976/malagasy-punctuation/resolve/main/labels.json
809 Bytes
| { | |
| "schema": "etiquette combinee = indice_casse * 7 + indice_ponct, lue sur le premier sous-jeton de chaque mot normalise", | |
| "cases": { | |
| "L": "minuscule", | |
| "C": "Capitale initiale", | |
| "U": "TOUT EN MAJUSCULES", | |
| "M": "mixte (forme lue dans tables.json['mixed'], sinon capitale apres la derniere apostrophe/trait d'union)" | |
| }, | |
| "puncts": [ | |
| "O", | |
| ",", | |
| ".", | |
| "?", | |
| "!", | |
| ":", | |
| ";" | |
| ], | |
| "normalisation_entree": "evaluate_asr.normalize (minuscules, [a-z0-9'-], apostrophes/tirets unifies)", | |
| "tables": "tables.json : 'mixed' = formes a casse mixte ; 'lex' = n-gramme normalise -> forme accentuee (ex. 'tan na' -> 'tanàna'), construite sur train + Wikipedia", | |
| "base": "Davlan/xlm-roberta-base-finetuned-malagasy", | |
| "entrainement": "work/punct/train.py --noise 0.5 --epochs 3 (checkpoint epoque 2.5)" | |
| } |