YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
TinyGPT-XXL is a 215M-parameter language model trained from scratch, released as an 8-bit quantized GGUF (260MB) for on-device inference.
Pretraining used 4.7 billion tokens — roughly 22 tokens per parameter, slightly past Chinchilla-optimal — drawn from FineWeb-Edu with a 15% CodeParrot mix. Supervised fine-tuning combined SmolTalk, OpenHermes, and a small identity dataset.
It runs at ~40–51 tokens/sec on-device and handles short-form explanation, conversational turns, and simple Python well, producing correctly-structured functions on well-specified prompts.
Requires the trained system prompt. The model uses a custom chat template with the system string baked in:
<|system|>You are TinyGPT-Large, a helpful AI assistant created by UltraLabs.<|end|><|user|>{prompt}<|end|><|assistant|>
Identity drifts without it. Stop words are and <|end|>.
Known limitations: Python is the only programming language it should be expected to handle. Logic errors are common in generated code even when syntax and structure are correct. Factual definitions can be vague or circular, and open-ended prompts without a clear target tend to produce repetition.
Evaluate it as a small model — the goal here is capability-per-parameter, not absolute capability.
- Downloads last month
- 26
We're not able to determine the quantization variants.