metadata
license: mit
datasets:
- VDC-team/DialoguesEN-2k
language:
- en
tags:
- VDC
- VDront
- LM
- LLM
- 20m
- english
SmallDront-20m
SmallDront-20m is a lightweight 20M parameter LM model fine-tuned for small talk in English. It improves upon our previous model, VDrontV2-mini, delivering more coherent and engaging conversations.
🧠 Model Details
- Architecture: GPT-style
- Parameters: 20,000,000
- Tokenizer: GPT2 tokenizer
- Special Tokens:
<|user|>and<|assistant|> - Training Data: DialoguesEN-2k
- Focus Topic: Small talk / casual conversation
✨ Key Improvements
- Better at handling greetings and opening lines
- More natural conversation flow
- Often asks follow-up questions (though sometimes absurd)
- Lower loss — example successful runs at loss 0.5
🗣️ Example Dialogues (loss 0.5)
You: Hey
Assistant: Hey there, what's the latest news in your world? What did you? ... (then follows info noise)
You: Hi
Assistant: Hello, got any plans for this day.
You: Hello!
Assistant: Hey! What's a favorite memory of yours? You? How about you?
Note: The model tends to generate informative noise after initial questions — a known tradeoff in this version.
🚀 Usage
Use use.py as a simple example for loading and interacting with the model.