Update README.md

README.md

@@ -1,216 +0,0 @@
----
-library_name: transformers
-license: mit
-base_model: xlm-roberta-base
-tags:
-- generated_from_trainer
-metrics:
-- precision
-- recall
-- f1
-- accuracy
-language:
-- multilingual
-- af
-- am
-- ar
-- as
-- ba
-- be
-- bg
-- bn
-- bo
-- br
-- bs
-- ca
-- ce
-- ckb
-- cs
-- cy
-- da
-- de
-- dv
-- el
-- en
-- eo
-- es
-- et
-- eu
-- fa
-- fi
-- fr
-- ga
-- gd
-- gl
-- gu
-- he
-- hi
-- hr
-- hu
-- hy
-- id
-- is
-- it
-- ja
-- jv
-- ka
-- kk
-- km
-- kn
-- ko
-- ku
-- ky
-- la
-- lb
-- lo
-- lt
-- lv
-- mg
-- mk
-- ml
-- mn
-- mr
-- ms
-- mt
-- my
-- ne
-- nl
-- 'no'
-- ny
-- oc
-- om
-- or
-- pa
-- pl
-- ps
-- pt
-- rm
-- ro
-- ru
-- sd
-- si
-- sk
-- sl
-- so
-- sq
-- sr
-- su
-- sv
-- sw
-- ta
-- te
-- tg
-- th
-- ti
-- tl
-- tr
-- tt
-- ug
-- uk
-- ur
-- uz
-- vi
-- yo
-- zh
-- zu
-model-index:
-- name: polyglot-tagger
-  results: []
-datasets:
-- wikimedia/wikipedia
-- HuggingFaceFW/finetranslations
-- google/smol
-- DerivedFunction/nlp-noise-snippets
-- DerivedFunction/wikipedia-language-snippets-filtered
-- DerivedFunction/finetranslations-filtered
-- DerivedFunction/lang-ner-v2
-- DerivedFunction/tatoeba-filtered
-pipeline_tag: token-classification
----
-
-
-![image](https://cdn-uploads.huggingface.co/production/uploads/67ee3f0a66388136438834cc/OnfV_fN2br5c4cPnOn6O0.png)
-
-
-Fine-tuned `xlm-roberta-base` for sentence-level language tagging across 100 languages.
-The model predicts BIO-style language tags over tokens, which makes it useful for
-language identification, code-switch detection, and multilingual document analysis.
-
-
-
-## Model description
-
-Introducing Polyglot Tagger, a new way to classify multi-lingual documents. By training specifically on token classification on individual sentences, the model 
-generalizes well on a variety of languages, while also behaves as a multi-label classifier, and extracts sentences based on its language. 
-
-## Intended uses & limitations
-This model can be treated as a base model for further fine-tuning on specific language identification extraction tasks. 
-Note that as a general language tagging model, it can potentially get confused from shared language families or from short texts. For example, English and German, Spanish and Portuguese, and Russian and Ukrainian.
-
-The model is trained on a sentence with a minimum of four tokens, so it may not accurately classify very short and ambigous statements. Note that this model is experimental
-and may produce unexpected results compared to generic text classifiers. It is trained on cleaned text, therefore, "messy" text may unexpectedly produce different results.
-
-> Note that Romanized versions of any language may only have minor representation in the training set, such as Romanized Russian, and Hindi.
-
-### Training and Evaluation Data
-A synthetic training row consists of 1-4 individual and mostly independent sentences extracted from various sources. The actual training and evaluation data, as well as coverage
-is found in `DerivedFunction/lang-ner-v2`.
-
-
-It achieves the following results on the evaluation set:
-- Loss: 0.0427
-- Precision: 0.8949
-- Recall: 0.9144
-- F1: 0.9046
-- Accuracy: 0.9892
-
-## Training procedure
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 5e-05
-- train_batch_size: 72
-- eval_batch_size: 36
-- seed: 42
-- gradient_accumulation_steps: 2
-- total_train_batch_size: 144
-- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
-- lr_scheduler_type: linear
-- num_epochs: 2
-- mixed_precision_training: Native AMP
-
-### Training results
-
-| Training Loss | Epoch  | Step  | Validation Loss | Precision | Recall | F1     | Accuracy |
-|:-------------:|:------:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
-| 0.0919        | 0.0894 | 2500  | 0.1243          | 0.7388    | 0.8336 | 0.7833 | 0.9712   |
-| 0.0798        | 0.1788 | 5000  | 0.0950          | 0.7928    | 0.8607 | 0.8254 | 0.9774   |
-| 0.0738        | 0.2682 | 7500  | 0.0857          | 0.8173    | 0.8722 | 0.8438 | 0.9785   |
-| 0.0611        | 0.3575 | 10000 | 0.0797          | 0.8247    | 0.8767 | 0.8499 | 0.9812   |
-| 0.0588        | 0.4469 | 12500 | 0.0732          | 0.8336    | 0.8843 | 0.8582 | 0.9822   |
-| 0.0542        | 0.5363 | 15000 | 0.0665          | 0.8560    | 0.8922 | 0.8737 | 0.9838   |
-| 0.0557        | 0.6257 | 17500 | 0.0613          | 0.8607    | 0.8949 | 0.8775 | 0.9845   |
-| 0.0486        | 0.7151 | 20000 | 0.0590          | 0.8567    | 0.8953 | 0.8755 | 0.9851   |
-| 0.0474        | 0.8045 | 22500 | 0.0601          | 0.8660    | 0.8971 | 0.8813 | 0.9854   |
-| 0.0545        | 0.8938 | 25000 | 0.0574          | 0.8675    | 0.9003 | 0.8836 | 0.9857   |
-| 0.0485        | 0.9832 | 27500 | 0.0566          | 0.8723    | 0.9018 | 0.8868 | 0.9858   |
-| 0.0440        | 1.0726 | 30000 | 0.0522          | 0.8769    | 0.9042 | 0.8904 | 0.9867   |
-| 0.0396        | 1.1620 | 32500 | 0.0509          | 0.8761    | 0.9046 | 0.8901 | 0.9873   |
-| 0.0383        | 1.2514 | 35000 | 0.0489          | 0.8788    | 0.9057 | 0.8921 | 0.9879   |
-| 0.0370        | 1.3408 | 37500 | 0.0486          | 0.8842    | 0.9087 | 0.8963 | 0.9877   |
-| 0.0350        | 1.4302 | 40000 | 0.0489          | 0.8769    | 0.9054 | 0.8909 | 0.9874   |
-| 0.0330        | 1.5195 | 42500 | 0.0478          | 0.8842    | 0.9091 | 0.8965 | 0.9879   |
-| 0.0308        | 1.6089 | 45000 | 0.0458          | 0.8897    | 0.9122 | 0.9008 | 0.9888   |
-| 0.0317        | 1.6983 | 47500 | 0.0454          | 0.8873    | 0.9114 | 0.8992 | 0.9887   |
-| 0.0322        | 1.7877 | 50000 | 0.0447          | 0.8900    | 0.9117 | 0.9007 | 0.9888   |
-| 0.0310        | 1.8771 | 52500 | 0.0439          | 0.8910    | 0.9126 | 0.9017 | 0.9888   |
-| 0.0294        | 1.9665 | 55000 | 0.0427          | 0.8949    | 0.9144 | 0.9046 | 0.9892   |
-
-
-### Framework versions
-
-- Transformers 5.0.0
-- Pytorch 2.10.0+cu128
-- Datasets 4.0.0
-- Tokenizers 0.22.2