AtomLlama-33K-5x5-DigitMesh

A minimal 33K parameter language model for 5×5 digit mesh recognition, built on the LlamaForCausalLM architecture.

Model Description

AtomLlama-33K-5x5-DigitMesh is an ultra-lightweight causal language model designed for efficient digit recognition from 5×5 binary mesh patterns. With only 33,000 parameters, this "atom-sized" model demonstrates effective pattern recognition with minimal computational resources.

Key Specifications

• Architecture: LlamaForCausalLM
• Parameters: ~33K
• Input: 5×5 binary mesh (25 tokens)
• Output: Digit tokens (D0-D9)
• Vocabulary Size: 14 tokens
• Context Length: 32 tokens
• Hidden Size: 32, Layers: 2, Attention Heads: 4

Quick Start

Serving with vLLM

python -m vllm.entrypoints.openai.api_server \
  --model junzzhu/atomllama-33K-5x5-DigitMesh \
  --max-model-len 32

Test Patterns

Example: Testing Digit 0

curl http://localhost:8000/v1/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "junzzhu/atomllama-33K-5x5-DigitMesh",
    "prompt": "1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 <SEP>",
    "max_tokens": 1,
    "temperature": 0
  }'

Expected output: D0

Test Patterns for Other Digits

Replace the prompt value in the curl command above with these patterns:

• Digit 1: "0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 <SEP>" → Expected: D1
• Digit 2: "1 1 1 1 0 0 0 0 1 1 0 1 1 1 0 1 1 0 0 0 1 1 1 1 1 <SEP>" → Expected: D2
• Digit 3: "1 1 1 1 0 0 0 0 1 1 0 1 1 1 0 0 0 0 1 1 1 1 1 1 0 <SEP>" → Expected: D3
• Digit 4: "1 0 0 0 1 1 0 0 0 1 1 1 1 1 1 0 0 0 0 1 0 0 0 0 1 <SEP>" → Expected: D4
• Digit 5: "1 1 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 1 1 1 <SEP>" → Expected: D5
• Digit 6: "1 1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 <SEP>" → Expected: D6
• Digit 7: "1 1 1 1 1 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 <SEP>" → Expected: D7
• Digit 8: "0 1 1 1 0 1 0 0 0 1 0 1 1 1 0 1 0 0 0 1 0 1 1 1 0 <SEP>" → Expected: D8
• Digit 9: "1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 <SEP>" → Expected: D9

Input Format

The model expects 25 space-separated binary values (0 or 1) representing a 5×5 grid, followed by <SEP>:

[5 values] [5 values] [5 values] [5 values] [5 values] <SEP>

Use Cases

• Educational demonstrations of minimal transformers
• Resource-constrained digit recognition
• Model compression research
• Pattern recognition proof-of-concepts

Limitations

• Fixed 5×5 resolution only
• Binary patterns only (no grayscale)
• No rotation or scaling support
• Digits 0-9 only

License

Apache-2.0

Citation

@misc{atomllama-33k-digitMesh,
  title={AtomLlama-33K-5x5-DigitMesh: A Minimal Parameter Model for Digit Recognition},
  author={Jun Zhu},
  year={2026}
  howpublished={\url{https://huggingface.co/junzzhu/atomllama-33K-5x5-DigitMesh/}}
}