File size: 1,774 Bytes
237b4e1 f6641df 237b4e1 f6641df 237b4e1 f6641df | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | ---
license: mit
language:
- pt
- en
pipeline_tag: text-generation
tags:
- pytorch
- causal-lm
- decoder-only
- educational
- small-language-model
---
# Delta Ultra Mini
Delta Ultra Mini 1.1 is a compact decoder-only language model created by Flame Corporation. It is intended as an educational small LLM release for studying tokenizer training, causal language modeling, checkpoints, and local generation.
## Model Details
- Architecture: decoder-only causal Transformer
- Parameters: about 124M
- Context length: 768 tokens
- Tokenizer: BPE
- License: MIT
## Intended Use
- Learning how small language models are structured and trained
- Running local inference experiments
- Building and testing custom small datasets
- Fine-tuning from a compact baseline
## Not Intended For
- Production assistant use without evaluation
- High-stakes decision making
- Security, medical, legal, or financial advice
- Claims of strong general reasoning
## Training Data
The seed dataset is small and conversational. It is designed to validate the training pipeline, not to produce a high-quality assistant on its own. Users are encouraged to create larger, cleaner datasets and train new checkpoints. The loader supports continuous text/Markdown plus structured JSONL, JSON, and CSV examples.
## Limitations
Delta Ultra Mini 1.1 can hallucinate, mix examples from the seed dataset, and answer incorrectly. The included checkpoint should be treated as an experimental baseline.
## Quick Start
```bash
pip install -r requirements.txt
python scripts/generate_delta.py --prompt "Quem e voce?" --checkpoint_path delta_checkpoint.pt --tokenizer_path tokenizer.json
```
## Citation
If you publish work using this model, cite the repository or model page where you downloaded it. |