TinyStories Llama2 1M (tinybpe1m) GGUF & HF Validation Suite

This repository provides ultra-lightweight Llama2 model files across various formats (both GGUF and Hugging Face / Safetensors), trained on the TinyStories dataset and optimized for testing and validation.

Why this repository exists

When developing a custom LLM inference engine, debugging with a full-sized model is slow. This suite offers a true 1M parameter scale model (~0.5MB to ~4MB depending on the quantization format), allowing developers to validate their loaders, serialization, quantization kernels, and inference logic step-by-step with maximum efficiency.

Difference from tiny1m

This is a BPE-based model variant. Unlike the standard tiny1m model, this model is NOT compatible with llama2.c. The custom SentencePiece BPE tokenizer utilized here relies on the byte_fallback mechanism to handle unknown characters. Because llama2.c's simplified native C loader/tokenizer cannot interpret or process byte_fallback routines, text generation will fail or corrupt in that environment. This suite is strictly designed and optimized for llama.cpp (GGUF) and Hugging Face transformers (Python) execution.

---

📂 Repository Structure & File Descriptions

1. GGUF Formats (Root Directory ./)

A comprehensive validation suite converted for llama.cpp and compatible engines. The tokenizer vocabulary and special tokens are fully embedded within each GGUF binary. Every compiled quantization variant available in the root directory is explicitly covered below:

Filename(s) / Wildcard Pattern	Type	Size	Purpose / Validation Target
tinybpe1m.F32.gguf	F32	~4.0 MB	Baseline Test. Validates GGUF parsing, tensor layout, matrix multiplication, RoPE, and Attention logic without dequantization overhead.
tinybpe1m.F16.gguf tinybpe1m.BF16.gguf	F16 BF16	~2.0 MB	Half-Precision Test. Validates 16-bit floating point loading, type casting, and inference stability.
tinybpe1m.Q8_0.gguf	Q8_0	~1.1 MB	Quantization Level 1. Validates block-based uniform scaling with 32 elements.
tinybpe1m.Q4_0.gguf tinybpe1m.Q4_1.gguf	Q4_0 Q4_1	~0.7 MB	Quantization Level 2. Validates classic 4-bit linear quantization and bit-unpacking logic.
tinybpe1m.Q2_K.gguf	Q2_K	~0.5 MB	Standard K-Quant (2-bit). Validates 2-bit super-block quantization parsing.
tinybpe1m.Q3_K_*.gguf ↳ tinybpe1m.Q3_K_S.gguf ↳ tinybpe1m.Q3_K_M.gguf ↳ tinybpe1m.Q3_K_L.gguf	Q3_K	~0.6 MB	Standard K-Quant (3-bit). Validates Small, Medium, and Large sub-variants of 3-bit multi-block structures.
tinybpe1m.Q4_K_*.gguf ↳ tinybpe1m.Q4_K_S.gguf ↳ tinybpe1m.Q4_K_M.gguf	Q4_K	~0.7 MB	Standard K-Quant (4-bit). Validates Small and Medium sub-variants of modern 4-bit super-block structural parsing.
tinybpe1m.Q5_K_*.gguf ↳ tinybpe1m.Q5_K_S.gguf ↳ tinybpe1m.Q5_K_M.gguf	Q5_K	~0.8 MB	Standard K-Quant (5-bit). Validates Small and Medium sub-variants of 5-bit mixed precision super-blocks.
tinybpe1m.Q6_K.gguf	Q6_K	~0.9 MB	Standard K-Quant (6-bit). Validates 6-bit high-fidelity super-block quantization.
tinybpe1m.IQ3_*.gguf ↳ tinybpe1m.IQ3_XXS.gguf ↳ tinybpe1m.IQ3_S.gguf	I-Quants	~0.5 MB	Importance Quants (3-bit). Non-linear 3-bit importance quantization targeting lookup table (codebook) decoding logic.
tinybpe1m.IQ4_*.gguf ↳ tinybpe1m.IQ4_NL.gguf ↳ tinybpe1m.IQ4_XS.gguf	I-Quants	~0.6 MB	Importance Quants (4-bit). Non-linear 4-bit importance quantization variants (Non-Linear and Extra Small).
tinybpe1m.TQ1_0.gguf tinybpe1m.TQ2_0.gguf	Ternary	~0.4 MB	Experimental. Ternary (-1, 0, 1) state quantization for cutting-edge engine testing.

2. Hugging Face Native Format (./hf/)

This directory contains the standard files required to load the model using the PyTorch transformers library:

• hf/model.safetensors: The raw, unquantized model weights stored securely in Safetensors format.
• hf/config.json: The architectural configuration file defining hyperparameters (layers, heads, dimensions).
• hf/generation_config.json: Default parameters optimized for text generation.
• hf/tokenizer_config.json: Tokenizer behavior layout enabling automatic BOS token injection and padding setup.
• hf/special_tokens_map.json: Architectural mappings tying token strings to exact internal special token IDs.
• hf/tokenizer.model: The custom 512-vocab SentencePiece tokenizer model file.

---

🚀 Usage Examples

A. Running GGUF via llama.cpp

To verify your local setup or test custom execution backends using the official native utilities:

./llama-cli -m tinybpe1m.Q4_K_M.gguf -p "Tom and Jerry are " -n 64 --temp 0.0

B. Loading Hugging Face Formats via Python

You can import the Hugging Face variant directly into Python using the transformers library.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

repo_id = "shibatch/tinybpe1m"

# The library automatically loads from the hf/ subfolder
tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="hf")
model = AutoModelForCausalLM.from_pretrained(repo_id, subfolder="hf")

prompt = "Tom and Jerry are "
inputs = tokenizer(prompt, return_tensors="pt")

with torch.no_grad():
    outputs = model.generate(
        **inputs, 
        max_new_tokens=64, 
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

📝 Model Specifications

The network architecture features an unshared output layer (lm_head) to keep memory structures consistent with standard Llama 2 definitions. Thanks to the highly optimized 512 vocabulary size, the token embedding and output layers remain extremely lightweight.

• Architecture: Llama 2 (Scaled-down variant)
• Dataset: TinyStories
• Total Parameters: ~1M (Exactly 896,256 parameters)
• Vocabulary Size: 512 (Custom SentencePiece BPE Tokenizer with byte_fallback enabled)
• Hidden Size (hidden_size): 128
• Number of Hidden Layers (num_hidden_layers): 4
• Number of Attention Heads (num_heads): 2
• Number of Key-Value Heads (num_kv_heads): 2
• Intermediate Size (intermediate_size): 352
• Max Position Embeddings (max_position_embeddings): 256

📜 Acknowledgments & License

• Original Implementation: Inspired by Andrej Karpathy's llama2.c project.
• Dataset: TinyStories dataset.
• License: MIT License. You are free to use, modify, and distribute these assets for any purpose.