Model Card: chrisjcc/mnist-vit
1. Model Details
- Model Architecture:
MNISTViT (Vision Transformer: patch size 4x4, 4 encoder layers, 4 attention heads, embed_dim 128)
- Framework: PyTorch (
2.15.0) + safetensors
- Trainable Parameters:
540,170 (6.25 MB full checkpoint / 2.07 MB SafeTensors weights)
- Input Contract:
(B, 1, 28, 28) float32 tensor normalized with $\mu = 0.1307$, $\sigma = 0.3081$
- Output Contract:
(B, 10) unnormalized class logits (0–9)
- Repository & Pipeline: Reproducible DVC + Hydra + Weights & Biases pipeline (
seed = 42)
- License: MIT
Sibling CNN-based model card: chrisjcc/mnist-cnn. Both models are available for comparative inference in the Streamlit demo space.
Layer-by-Layer Architecture
| Stage |
Layer Composition |
Output Shape |
| Input |
Normalized Grayscale Digit Image |
(B, 1, 28, 28) |
| Patch Extraction |
Non-overlapping 4x4 Patches ($N = (28/4)^2 = 49$) |
(B, 49, 16) |
| Linear Projection |
Linear(16 -> 128) + Learnable [CLS] Token |
(B, 50, 128) |
| Position Embedding |
1D Learnable Positional Embeddings + Dropout(p=0.1) |
(B, 50, 128) |
| Transformer Encoder |
4 x Transformer Blocks (LayerNorm -> MultiheadAttention(4 heads) -> LayerNorm -> MLP(dim=256)) |
(B, 50, 128) |
| Classifier Head |
LayerNorm -> Extract [CLS] Token -> Linear(128 -> 10) |
(B, 10) |
2. Intended Use & Limitations
Intended Use
- Handwritten single-digit classification (
0–9) on $28 \times 28$ grayscale images.
- Reference implementation for reproducible MLOps pipelines combining Hydra, DVC, Weights & Biases, Hugging Face Hub, and Hugging Face Docker Spaces.
- Interactive educational demonstrations via the bundled Streamlit application.
Out-of-Scope Use & Limitations
- Not designed for multi-digit OCR, alphanumeric text recognition, or natural scene text detection (SVHN/ICDAR).
- Sensitive to foreground/background polarity: MNIST digits are white strokes on a black background. Real-world photos of dark ink on white paper require polarity inversion (
invert=True in preprocess_pil_image).
3. Training Methodology & Hyperparameters
| Parameter |
Value |
| Dataset |
ylecun/mnist (54,000 train / 6,000 stratified val / 10,000 held-out test) |
| Global Seed |
42 (deterministic = True) |
| Optimizer |
adam (lr = 0.001, weight_decay = 0.0) |
| Epochs |
5 (early stopping patience = None) |
| Batch Size |
128 (train) / 256 (eval) |
| Hardware Device |
cpu |
4. Evaluation Results (Held-Out Test Set, $N = 10000$)
| Metric |
Value |
| Test Accuracy |
0.9485 (94.85%) |
| Test Cross-Entropy Loss |
0.1581 |
| Macro Precision |
0.9494 |
| Macro Recall |
0.9474 |
| Macro F1-Score |
0.9480 |
| Weighted F1-Score |
0.9485 |
Per-Class Performance (0–9)
| Digit Class |
Precision |
Recall |
F1-Score |
Support |
| Digit 0 |
0.9542 |
0.9786 |
0.9662 |
980 |
| Digit 1 |
0.9850 |
0.9868 |
0.9859 |
1135 |
| Digit 2 |
0.9634 |
0.9438 |
0.9535 |
1032 |
| Digit 3 |
0.9040 |
0.9604 |
0.9313 |
1010 |
| Digit 4 |
0.9758 |
0.9430 |
0.9591 |
982 |
| Digit 5 |
0.9699 |
0.9036 |
0.9356 |
892 |
| Digit 6 |
0.9694 |
0.9593 |
0.9643 |
958 |
| Digit 7 |
0.9099 |
0.9728 |
0.9403 |
1028 |
| Digit 8 |
0.9504 |
0.9055 |
0.9274 |
974 |
| Digit 9 |
0.9117 |
0.9207 |
0.9162 |
1009 |
5. Inference Latency & Throughput Benchmarks
Measured over 50 timed forward passes following 10 warm-up runs on device cpu:
| Batch Size |
Mean Latency (ms) |
Std (ms) |
P50 Latency (ms) |
P95 Latency (ms) |
P99 Latency (ms) |
Throughput (samples/sec) |
| 1 |
0.831 |
0.300 |
0.712 |
1.317 |
2.013 |
1203.9 |
| 32 |
7.768 |
0.722 |
7.630 |
9.174 |
10.005 |
4119.7 |
| 128 |
30.236 |
2.708 |
29.503 |
33.610 |
40.826 |
4233.3 |
6. How to Load & Run Inference
from PIL import Image
from mnist_pipeline.inference import load_inference_model, predict_pil_image
model, source = load_inference_model("chrisjcc/mnist-vit", device="cpu")
image = Image.open("digit_sample.png")
result = predict_pil_image(model, image, device="cpu")
print(f"Predicted Digit: {result.predicted_digit} (Confidence: {result.confidence:.2%})")