| --- |
| license: mit |
| datasets: |
| - VDC-team/DialoguesEN-2k |
| language: |
| - en |
| tags: |
| - VDC |
| - VDront |
| - LM |
| - LLM |
| - 20m |
| - english |
| --- |
| |
| # SmallDront-20m |
|  |
|
|
| **SmallDront-20m** is a lightweight 20M parameter LM model fine-tuned for small talk in English. It improves upon our previous model, `VDrontV2-mini`, delivering more coherent and engaging conversations. |
|
|
| ## 🧠 Model Details |
|
|
| - **Architecture:** GPT-style |
| - **Parameters:** 20,000,000 |
| - **Tokenizer:** GPT2 tokenizer |
| - **Special Tokens:** `<|user|>` and `<|assistant|>` |
| - **Training Data:** [DialoguesEN-2k](https://huggingface.co/datasets/VDC-team/DialoguesEN-2k) |
| - **Focus Topic:** Small talk / casual conversation |
|
|
| ## ✨ Key Improvements |
|
|
| - Better at handling greetings and opening lines |
| - More natural conversation flow |
| - Often asks follow-up questions (though sometimes absurd) |
| - Lower loss — example successful runs at loss **0.5** |
|
|
| ## 🗣️ Example Dialogues (loss 0.5) |
|
|
| > **You:** Hey |
| > **Assistant:** Hey there, what's the latest news in your world? What did you? ... *(then follows info noise)* |
|
|
| > **You:** Hi |
| > **Assistant:** Hello, got any plans for this day. |
|
|
| > **You:** Hello! |
| > **Assistant:** Hey! What's a favorite memory of yours? You? How about you? |
|
|
| *Note: The model tends to generate informative noise after initial questions — a known tradeoff in this version.* |
|
|
| ## 🚀 Usage |
|
|
| Use `use.py` as a simple example for loading and interacting with the model. |
|
|
| ### HuggingFace Format |
|
|
|
|
|
|
|
|
