| ---
|
| license: mit
|
| library_name: sklearn
|
| tags:
|
| - language-identification
|
| - text-classification
|
| - tfidf
|
| language:
|
| - multilingual
|
| pipeline_tag: text-classification
|
| ---
|
|
|
| # Language Detector (TF-IDF char n-grams + linear SVM Classifier)
|
|
|
| Module 1 of the RAG-Based Mental Health Support Chatbot. Predicts the language
|
| code of a short text (e.g. `en`, `fr`, `ar`).
|
|
|
| ## Files
|
|
|
| - `vectorizer.joblib` — fitted `TfidfVectorizer` (character n-grams)
|
| - `classifier.joblib` — fitted linear classifier
|
|
|
| ## Usage
|
|
|
| ```python
|
| import joblib
|
| from huggingface_hub import hf_hub_download
|
|
|
| vec = joblib.load(hf_hub_download("GAD01/svm-language-detector", "vectorizer.joblib"))
|
| clf = joblib.load(hf_hub_download("GAD01/svm-language-detector", "classifier.joblib"))
|
| print(clf.predict(vec.transform(["Bonjour, comment allez-vous ?"]))[0]) # -> "fr"
|
| ```
|
|
|
| Trained on [papluca/language-identification](https://huggingface.co/datasets/papluca/language-identification).
|
|
|
