---
license: mit
language:
  - pt
  - en
pipeline_tag: text-generation
tags:
  - pytorch
  - causal-lm
  - decoder-only
  - educational
  - small-language-model
---

# Delta Ultra Mini

Delta Ultra Mini is a compact decoder-only language model created by Flame Corporation. It is intended as an educational small LLM release for studying tokenizer training, causal language modeling, checkpoints, and local generation.

## Model Details

- Architecture: decoder-only causal Transformer
- Parameters: about 50M
- Context length: 512 tokens
- Tokenizer: BPE
- License: MIT

## Intended Use

- Learning how small language models are structured and trained
- Running local inference experiments
- Building and testing custom small datasets
- Fine-tuning from a compact baseline

## Not Intended For

- Production assistant use without evaluation
- High-stakes decision making
- Security, medical, legal, or financial advice
- Claims of strong general reasoning

## Training Data

The seed dataset is small and conversational. It is designed to validate the training pipeline, not to produce a high-quality assistant on its own. Users are encouraged to create larger, cleaner datasets and train new checkpoints.

## Limitations

Delta Ultra Mini can hallucinate, mix examples from the seed dataset, and answer incorrectly. The included checkpoint should be treated as an experimental baseline.

## Quick Start

```bash
pip install -r requirements.txt
python scripts/generate_delta.py --prompt "Quem e voce?" --checkpoint_path delta_checkpoint.pt --tokenizer_path tokenizer.json
```

## Citation

If you publish work using this model, cite the repository or model page where you downloaded it.