| --- |
| license: mit |
| language: |
| - pt |
| - en |
| pipeline_tag: text-generation |
| tags: |
| - pytorch |
| - causal-lm |
| - decoder-only |
| - educational |
| - small-language-model |
| --- |
| |
| # Delta Ultra Mini |
|
|
| Delta Ultra Mini is a compact decoder-only language model created by Flame Corporation. It is intended as an educational small LLM release for studying tokenizer training, causal language modeling, checkpoints, and local generation. |
|
|
| ## Model Details |
|
|
| - Architecture: decoder-only causal Transformer |
| - Parameters: about 50M |
| - Context length: 512 tokens |
| - Tokenizer: BPE |
| - License: MIT |
|
|
| ## Intended Use |
|
|
| - Learning how small language models are structured and trained |
| - Running local inference experiments |
| - Building and testing custom small datasets |
| - Fine-tuning from a compact baseline |
|
|
| ## Not Intended For |
|
|
| - Production assistant use without evaluation |
| - High-stakes decision making |
| - Security, medical, legal, or financial advice |
| - Claims of strong general reasoning |
|
|
| ## Training Data |
|
|
| The seed dataset is small and conversational. It is designed to validate the training pipeline, not to produce a high-quality assistant on its own. Users are encouraged to create larger, cleaner datasets and train new checkpoints. |
|
|
| ## Limitations |
|
|
| Delta Ultra Mini can hallucinate, mix examples from the seed dataset, and answer incorrectly. The included checkpoint should be treated as an experimental baseline. |
|
|
| ## Quick Start |
|
|
| ```bash |
| pip install -r requirements.txt |
| python scripts/generate_delta.py --prompt "Quem e voce?" --checkpoint_path delta_checkpoint.pt --tokenizer_path tokenizer.json |
| ``` |
|
|
| ## Citation |
|
|
| If you publish work using this model, cite the repository or model page where you downloaded it. |
|
|
