Instructions to use msislam/code-mixed-language-detection-XLMRoberta with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use msislam/code-mixed-language-detection-XLMRoberta with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="msislam/code-mixed-language-detection-XLMRoberta")# Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("msislam/code-mixed-language-detection-XLMRoberta") model = AutoModelForTokenClassification.from_pretrained("msislam/code-mixed-language-detection-XLMRoberta", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -20,7 +20,7 @@ widget:
|
|
| 20 |
# Code-Mixed Language Detection using XLM-RoBERTa
|
| 21 |
|
| 22 |
## Description
|
| 23 |
-
This model detects languages in a
|
| 24 |
|
| 25 |
## Training Dataset
|
| 26 |
The training dataset is based on [The Multilingual Amazon Reviews Corpus](https://huggingface.co/datasets/amazon_reviews_multi). The preprocessed dataset that has been used to train, validate, and test this model can be found [here](https://huggingface.co/datasets/msislam/marc-code-mixed-small).
|
|
|
|
| 20 |
# Code-Mixed Language Detection using XLM-RoBERTa
|
| 21 |
|
| 22 |
## Description
|
| 23 |
+
This model detects languages in a Code-Mixed text with their boundaries by classifying each token. Currently, it supports German (DE), English (EN), Spanish (ES), and French (FR) languages. The model is fine-tuned on [xlm-roberta-base](https://huggingface.co/xlm-roberta-base).
|
| 24 |
|
| 25 |
## Training Dataset
|
| 26 |
The training dataset is based on [The Multilingual Amazon Reviews Corpus](https://huggingface.co/datasets/amazon_reviews_multi). The preprocessed dataset that has been used to train, validate, and test this model can be found [here](https://huggingface.co/datasets/msislam/marc-code-mixed-small).
|