---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
library_name: transformers
tags:
- i3-arhitecture
---

# i3-tiny

**i3-tiny** is a compact, efficient character-level language model designed for experimentation and exploration in text generation. Despite its small size, it packs a surprising punch for creative and research-oriented tasks, generating sequences that are quirky, unpredictable, and full of “human-like” character-level errors.

---

## Model Overview

i3-tiny is trained to predict the next character in a sequence, making it ideal for **character-level language modeling**, **creative text generation**, and **research on lightweight, efficient models**. Its small footprint allows rapid experimentation, even on modest hardware, and it provides a playground for studying how models learn patterns in sequences of characters.

The model is **intentionally experimental** — it’s not aligned, fact-checked, or polished. Instead, it showcases how a compact architecture can capture patterns in text, learn from repetition, and generate outputs that are sometimes surprisingly coherent, sometimes hilariously garbled.

---

## Training Details

* **Dataset:** ~45,830 characters (a curated text corpus repeated to improve exposure)
* **Vocabulary:** 34 characters (all lowercased)
* **Sequence length:** 128
* **Training iterations:** 2,000
* **Batch size:** 2
* **Optimizer:** AdamW, learning rate 3e-4
* **Model parameters:** 711,106
* **Performance notes:** Each iteration takes roughly 400–500 ms; 100 iterations take ~45 s on average. Loss steadily decreased from 3.53 to 2.15 over training.

**Example generation (iteration 1200):**

```
Prompt: "The quick"
Generated: the quick efehn. dethe cans the fice the fpeens antary of eathetint, an thadat hitimes the and cow thig, and
```

These outputs capture the **chaotic creativity** of a character-level model: a mixture of readable words, invented forms, and surprising sequences.

---

## Intended Uses

* **Character-level text generation experiments**
* **Research and education**: studying lightweight language models, sequence learning, and text modeling
* **Creative exploration**: generating quirky text or procedural content for games, demos, or artistic projects

> ⚠️ i3-tiny is experimental and **not intended for production or high-stakes applications**. Text may be repetitive, nonsensical, or inconsistent.

---

## Limitations

* Small vocabulary and character-level modeling limit natural language fluency
* Outputs are **highly experimental** and not fact-checked
* Generated sequences can be repetitive or unexpectedly garbled
* Not aligned or safety-checked

---

## Model Weights

* Stored in `model.bin`
* Compatible with PyTorch