Model Card: chrisjcc/mnist-vit

1. Model Details

  • Model Architecture: MNISTViT (Vision Transformer: patch size 4x4, 4 encoder layers, 4 attention heads, embed_dim 128)
  • Framework: PyTorch (2.15.0) + safetensors
  • Trainable Parameters: 540,170 (6.25 MB full checkpoint / 2.07 MB SafeTensors weights)
  • Input Contract: (B, 1, 28, 28) float32 tensor normalized with $\mu = 0.1307$, $\sigma = 0.3081$
  • Output Contract: (B, 10) unnormalized class logits (0–9)
  • Repository & Pipeline: Reproducible DVC + Hydra + Weights & Biases pipeline (seed = 42)
  • License: MIT

Sibling CNN-based model card: chrisjcc/mnist-cnn. Both models are available for comparative inference in the Streamlit demo space.

Layer-by-Layer Architecture

Stage Layer Composition Output Shape
Input Normalized Grayscale Digit Image (B, 1, 28, 28)
Patch Extraction Non-overlapping 4x4 Patches ($N = (28/4)^2 = 49$) (B, 49, 16)
Linear Projection Linear(16 -> 128) + Learnable [CLS] Token (B, 50, 128)
Position Embedding 1D Learnable Positional Embeddings + Dropout(p=0.1) (B, 50, 128)
Transformer Encoder 4 x Transformer Blocks (LayerNorm -> MultiheadAttention(4 heads) -> LayerNorm -> MLP(dim=256)) (B, 50, 128)
Classifier Head LayerNorm -> Extract [CLS] Token -> Linear(128 -> 10) (B, 10)

2. Intended Use & Limitations

Intended Use

  • Handwritten single-digit classification (0–9) on $28 \times 28$ grayscale images.
  • Reference implementation for reproducible MLOps pipelines combining Hydra, DVC, Weights & Biases, Hugging Face Hub, and Hugging Face Docker Spaces.
  • Interactive educational demonstrations via the bundled Streamlit application.

Out-of-Scope Use & Limitations

  • Not designed for multi-digit OCR, alphanumeric text recognition, or natural scene text detection (SVHN/ICDAR).
  • Sensitive to foreground/background polarity: MNIST digits are white strokes on a black background. Real-world photos of dark ink on white paper require polarity inversion (invert=True in preprocess_pil_image).

3. Training Methodology & Hyperparameters

Parameter Value
Dataset ylecun/mnist (54,000 train / 6,000 stratified val / 10,000 held-out test)
Global Seed 42 (deterministic = True)
Optimizer adam (lr = 0.001, weight_decay = 0.0)
Epochs 5 (early stopping patience = None)
Batch Size 128 (train) / 256 (eval)
Hardware Device cpu

4. Evaluation Results (Held-Out Test Set, $N = 10000$)

Metric Value
Test Accuracy 0.9485 (94.85%)
Test Cross-Entropy Loss 0.1581
Macro Precision 0.9494
Macro Recall 0.9474
Macro F1-Score 0.9480
Weighted F1-Score 0.9485

Per-Class Performance (0–9)

Digit Class Precision Recall F1-Score Support
Digit 0 0.9542 0.9786 0.9662 980
Digit 1 0.9850 0.9868 0.9859 1135
Digit 2 0.9634 0.9438 0.9535 1032
Digit 3 0.9040 0.9604 0.9313 1010
Digit 4 0.9758 0.9430 0.9591 982
Digit 5 0.9699 0.9036 0.9356 892
Digit 6 0.9694 0.9593 0.9643 958
Digit 7 0.9099 0.9728 0.9403 1028
Digit 8 0.9504 0.9055 0.9274 974
Digit 9 0.9117 0.9207 0.9162 1009

5. Inference Latency & Throughput Benchmarks

Measured over 50 timed forward passes following 10 warm-up runs on device cpu:

Batch Size Mean Latency (ms) Std (ms) P50 Latency (ms) P95 Latency (ms) P99 Latency (ms) Throughput (samples/sec)
1 0.831 0.300 0.712 1.317 2.013 1203.9
32 7.768 0.722 7.630 9.174 10.005 4119.7
128 30.236 2.708 29.503 33.610 40.826 4233.3

6. How to Load & Run Inference

from PIL import Image
from mnist_pipeline.inference import load_inference_model, predict_pil_image

# Download from Hugging Face Hub (or load from local fallback checkpoint)
model, source = load_inference_model("chrisjcc/mnist-vit", device="cpu")

# Load any handwritten digit image
image = Image.open("digit_sample.png")
result = predict_pil_image(model, image, device="cpu")

print(f"Predicted Digit: {result.predicted_digit} (Confidence: {result.confidence:.2%})")
Downloads last month
48
Safetensors
Model size
540k params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Datasets used to train chrisjcc/mnist-vit

Space using chrisjcc/mnist-vit 1

Evaluation results