---
license: mit
datasets:
- VDC-team/DialoguesEN-2k
language:
- en
tags:
- VDC
- VDront
- LM
- LLM
- 20m
- english
---

# SmallDront-20m
![](preview.jpg)

**SmallDront-20m** is a lightweight 20M parameter LM model fine-tuned for small talk in English. It improves upon our previous model, `VDrontV2-mini`, delivering more coherent and engaging conversations.

## 🧠 Model Details

- **Architecture:** GPT-style  
- **Parameters:** 20,000,000  
- **Tokenizer:** GPT2 tokenizer  
- **Special Tokens:** `<|user|>` and `<|assistant|>`  
- **Training Data:** [DialoguesEN-2k](https://huggingface.co/datasets/VDC-team/DialoguesEN-2k)  
- **Focus Topic:** Small talk / casual conversation  

## ✨ Key Improvements

- Better at handling greetings and opening lines  
- More natural conversation flow  
- Often asks follow-up questions (though sometimes absurd)  
- Lower loss — example successful runs at loss **0.5**

## 🗣️ Example Dialogues (loss 0.5)

> **You:** Hey  
> **Assistant:** Hey there, what's the latest news in your world? What did you? ... *(then follows info noise)*

> **You:** Hi  
> **Assistant:** Hello, got any plans for this day.

> **You:** Hello!  
> **Assistant:** Hey! What's a favorite memory of yours? You? How about you?

*Note: The model tends to generate informative noise after initial questions — a known tradeoff in this version.*

## 🚀 Usage

Use `use.py` as a simple example for loading and interacting with the model.

### HuggingFace Format