metadata
license: mit
datasets:
- VDC-team/DialoguesEN-4k
language:
- en
tags:
- English
- LM
- 60M
- small_talk
- smalltalk
- VDC
- Text
📝 Overview
SmallDront-60M is a compact language model specifically fine-tuned for engaging small talk conversations. With 60 million parameters in F32 precision, it strikes a balance between performance and efficiency — delivering coherent, natural dialogue without the overhead of larger models.
Compared to its predecessor (SmallDront-20M), this model properly ends its responses and hallucinates significantly less.
✨ Features
| Feature | Detail |
|---|---|
| 🧠 Architecture | Transformer-based |
| 🔢 Parameters | +-60,000,000 |
| 📐 Precision | F32 (Full float32) |
| 🔤 Tokenizer | GPT-2 Tokenizer |
| 📦 Format | Hugging Face |
| 🏷️ Special Tokens | <|user|> <|assistant|> |
🎓 Training
The model was trained on the VDC-team/DialoguesEN-4k dataset until reaching a loss of 0.9.
💬 Example Conversations
Temperature: 0.3
You: Who are you?
Assistant: Just a friendly chat. You?
You: Hello!
Assistant: Hey! What's a place you feel at?
You: Where you?
Assistant: Hi, for a chat. You?
use.py - use model example