| ---
|
| license: apache-2.0
|
| base_model: Qwen/Qwen2.5-0.5B-Instruct
|
| tags:
|
| - language-identification
|
| - knowledge-distillation
|
| - dcube
|
| - qwen2
|
| pipeline_tag: text-generation
|
| ---
|
|
|
| # DCUBE Language Identifier
|
|
|
| Fine-tuned **Qwen2.5-0.5B-Instruct** for multilingual language identification via knowledge distillation.
|
|
|
| ## Usage
|
|
|
| **Paste any text** — the embedded chat template automatically:
|
| 1. Applies the DCUBE system prompt
|
| 2. Wraps your input with `Identify the language:`
|
|
|
| ### LM Studio
|
| 1. Download the **GGUF** file (or convert from this repo)
|
| 2. Load the model — **no manual system prompt needed** if the GGUF includes the chat template
|
| 3. Paste text directly, e.g. `Обичам да слушам музика вечер`
|
| 4. Set **Temperature = 0**
|
|
|
| ### Python (llama-cpp)
|
| ```bash
|
| py run_gguf.py "Your text here"
|
| ```
|
|
|
| ### Hugging Face Transformers
|
| ```python
|
| from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
| tokenizer = AutoTokenizer.from_pretrained("YOUR_REPO_ID", trust_remote_code=True)
|
| model = AutoModelForCausalLM.from_pretrained("YOUR_REPO_ID", trust_remote_code=True)
|
|
|
| messages = [{"role": "user", "content": "Le temps est magnifique."}]
|
| prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
| # Template auto-adds system prompt + "Identify the language:" prefix
|
| ```
|
|
|
| ## Supported languages
|
|
|
| Arabic, Bulgarian, German, Greek, English, Spanish, French, Hindi, Italian, Japanese, Dutch, Polish, Portuguese, Russian, Swahili, Thai, Turkish, Urdu, Vietnamese, Chinese, Malayalam
|
|
|
| ## System prompt (embedded in chat template)
|
|
|
| ```
|
| You are DCUBE Language Identifier, a language identification assistant created by DCUBE Ai (www.dcubeai.com). Given a text, respond with ONLY the language name. Do not include any explanation.
|
| ```
|
|
|
| ## Training
|
|
|
| - Teacher: Qwen3-14B (knowledge distillation)
|
| - Method: LoRA SFT on papluca/language-identification + Malayalam samples
|
| - DCUBE Ai · www.dcubeai.com
|
|
|
