README.md

---
license: mit
datasets:
- VDC-team/DialoguesEN-4k
language:
- en
tags:
- English
- LM
- 60M
- small_talk
- smalltalk
- VDC
- Text
---
![](preview.jpg)
<div align="center">

# 🤖 SmallDront-60M

*A lightweight conversational AI model designed for natural small talk*

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Hugging Face](https://img.shields.io/badge/🤗%20Hugging%20Face-Model-blue)](https://huggingface.co/)
[![Parameters](https://img.shields.io/badge/Parameters-60M-green)]()
[![Format](https://img.shields.io/badge/Format-F32-orange)]()

</div>

---

## 📝 Overview

**SmallDront-60M** is a compact language model specifically fine-tuned for engaging small talk conversations. With 60 million parameters in F32 precision, it strikes a balance between performance and efficiency — delivering coherent, natural dialogue without the overhead of larger models.

> *Compared to its predecessor (SmallDront-20M), this model properly ends its responses and hallucinates significantly less.*

---

## ✨ Features

| Feature | Detail |
|---------|--------|
| 🧠 **Architecture** | Transformer-based |
| 🔢 **Parameters** | +-60,000,000 |
| 📐 **Precision** | F32 (Full float32) |
| 🔤 **Tokenizer** | **GPT-2 Tokenizer** |
| 📦 **Format** | Hugging Face  |
| 🏷️ **Special Tokens** | `<\|user\|>` `<\|assistant\|>` |

---

## 🎓 Training

The model was trained on the **[VDC-team/DialoguesEN-4k](https://huggingface.co/datasets/VDC-team/DialoguesEN-4k)** dataset until reaching a loss of **0.9**.

---

## 💬 Example Conversations

*Temperature: 0.3*

```text
You: Who are you?
Assistant: Just a friendly chat. You?

You: Hello!
Assistant: Hey! What's a place you feel at?

You: Where you?
Assistant: Hi, for a chat. You?

```

**use.py** - use model example