README.md

---
license: apache-2.0
language:
- en
library_name: transformers
datasets:
- HuggingFaceFW/fineweb
tags:
- text-generation
- causal-lm
- custom-code
pipeline_tag: text-generation
---

# OLM3 Nano

OLM3 Nano is a small (~1B parameter) decoder-only causal language model, trained from scratch on the [FineWeb](https://huggingface.co/datasets/HuggingFaceFW/fineweb) corpus.

## Model details

| | |
|---|---|
| Architecture | Decoder-only Transformer with RoPE positional embeddings |
| Parameters | ~1.02B |
| Hidden size | 2048 |
| Layers | 16 |
| Attention heads | 16 |
| Vocabulary size | 50304 |
| Max context length | 2048 tokens |
| Positional encoding | Rotary (RoPE), θ = 10000 |
| Normalization | RMSNorm |
| Weight tying | Input embeddings and output (LM head) are tied |
| Training data | FineWeb |
| Checkpoint step | 14086 |

## Tokenizer

This model was trained with the **GPT-2 tokenizer** (as used by [`tiktoken`](https://github.com/openai/tiktoken)'s `"gpt2"` encoding). Use `GPT2TokenizerFast` / `AutoTokenizer` from this repo, or `tiktoken.get_encoding("gpt2")` directly.

## Usage

This model uses custom modeling code, so `trust_remote_code=True` is required.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "MedcellStudios/OLM3Nano"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.float32,
).to("cuda")
model.eval()

prompt = "Hello! How are you?"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to("cuda")

with torch.no_grad():
    output = model.generate(
        input_ids,
        max_new_tokens=80,
        do_sample=True,
        temperature=0.8,
        top_k=40,
        repetition_penalty=1.2,
        no_repeat_ngram_size=3,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(output[0, input_ids.shape[1]:], skip_special_tokens=True))
```

## Intended use and limitations

OLM3 Nano is a small research/hobby-scale language model. It is **not instruction-tuned or aligned**, and its outputs should not be treated as factual, safe, or suitable for production use without further fine-tuning and evaluation. Given its small parameter count and training scale, expect frequent repetition, factual errors, and limited reasoning ability compared to larger models.

### Known issue: over-memorized personality section

After training and deploying this model on our website, we noticed that the model had **over-memorized the personality section of its SFT data**. As a result, some responses can be inconsistent — the model may repeat fixed personality-related phrasing verbatim rather than generating a natural, context-appropriate reply. We're aware of this and plan to address it in a future fine-tuning pass with more varied personality examples; in the meantime, treat personality-flavored outputs with some skepticism.

## License

Apache 2.0.