metadata
license: mit
language:
- pt
- en
pipeline_tag: text-generation
tags:
- pytorch
- causal-lm
- decoder-only
- educational
- small-language-model
Delta Ultra Mini
Delta Ultra Mini is a compact decoder-only language model created by Flame Corporation. It is intended as an educational small LLM release for studying tokenizer training, causal language modeling, checkpoints, and local generation.
Model Details
- Architecture: decoder-only causal Transformer
- Parameters: about 50M
- Context length: 512 tokens
- Tokenizer: BPE
- License: MIT
Intended Use
- Learning how small language models are structured and trained
- Running local inference experiments
- Building and testing custom small datasets
- Fine-tuning from a compact baseline
Not Intended For
- Production assistant use without evaluation
- High-stakes decision making
- Security, medical, legal, or financial advice
- Claims of strong general reasoning
Training Data
The seed dataset is small and conversational. It is designed to validate the training pipeline, not to produce a high-quality assistant on its own. Users are encouraged to create larger, cleaner datasets and train new checkpoints.
Limitations
Delta Ultra Mini can hallucinate, mix examples from the seed dataset, and answer incorrectly. The included checkpoint should be treated as an experimental baseline.
Quick Start
pip install -r requirements.txt
python scripts/generate_delta.py --prompt "Quem e voce?" --checkpoint_path delta_checkpoint.pt --tokenizer_path tokenizer.json
Citation
If you publish work using this model, cite the repository or model page where you downloaded it.