Instructions to use meatloafer/efficientnet-b0-stage-attention with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Keras
How to use meatloafer/efficientnet-b0-stage-attention with Keras:
# !pip install -U keras tensorflow huggingface_hub # Keras needs TensorFlow installed to read "hf://" paths, so the tensorflow backend is selected here; # "jax" and "torch" also work for computation once TensorFlow is installed. import os os.environ["KERAS_BACKEND"] = "tensorflow" import keras model = keras.saving.load_model("hf://meatloafer/efficientnet-b0-stage-attention") - Notebooks
- Google Colab
- Kaggle
EfficientNet-B0 Sparse Stage Attention ($P_3/P_4/P_5$) & Alternating Weight Sharing ($[B_1, A, B, A]$)
Official model weights, reference architecture builders, and SHA-256 verification manifests for the paper:
Where Global Context and Weight Sharing Pay Off in Compact Vision Backbones
Michael Schroeder (2026)
Overview
This repository provides the trained checkpoints and model-construction modules for two complementary structural modifications to EfficientNet-B0 evaluated across the full 50,000-image ImageNet-1k validation split:
Sparse Zero-Gated Stage Attention ($P_3/P_4/P_5$): Instead of wrapping the backbone in a heavy multi-scale feature pyramid (BiFPN) or replacing entire convolutional stages with Transformer blocks, we preserve the entire MBConv trunk and insert lightweight Pre-LN residual self-attention adapters with zero-initialized gates at three semantic scales:
- $P_3$ ($28 \times 28$, $40\text{ ch}$): $7 \times 7$ non-overlapping windowed multi-head self-attention (MHSA)
- $P_4$ ($14 \times 14$, $112\text{ ch}$): global MHSA
- $P_5$ ($7 \times 7$, $320\text{ ch}$): global MHSA
Under a strictly controlled 88-epoch schedule consuming bit-for-bit identical stateless image batches from the same initial backbone, Sparse Stage Attention outperforms Plain-B0 at every validation milestone, finishing $+0.954$ percentage points ahead ($73.878%$ vs. $72.924%$, and $74.020%$ in standalone training). Learned gate activity increases monotonically with semantic depth ($27.2% \rightarrow 45.4% \rightarrow 66.3%$ relative update-to-input RMS from $P_3 \rightarrow P_4 \rightarrow P_5$).
Alternating Recurrent Weight Sharing ($[B_1, A, B, A]$): Tying repeated inverted-residual blocks within Stage 6 to an alternating two-template cycle ($[B_1, A, B, A]$, where Block 4 reuses Block 2's convolutional/SE weights $A$ while retaining pass-specific BatchNorm statistics that add only $+1,536$ parameters, $<0.03%$ of the model) reduces total model parameters by $10.9%$ ($582,384$ weights) and recovers $+10.82$ percentage points in a single continuation epoch ($5,004$ steps) to **$70.92%$**βoutperforming single-template tying ($[B_1, A, A, A]$, $66.10%$) by $+4.82$ percentage points.
Included Checkpoints (weights/)
All checkpoints are evaluated on the complete 50,000-image ImageNet-1k validation set (224x224 resolution, standard 88-epoch ReLU6/SGD baseline protocol).
| File | Architecture / Arm | Params | Top-1 (%) | Top-5 (%) | Size | SHA-256 Prefix |
|---|---|---|---|---|---|---|
weights/sparse_attention_p3p4p5_standalone_e88.weights.h5 |
Sparse Stage Attention ($P_3/P_4/P_5$) β Standalone 88-ep recovery (e88) | 6.27M | 74.020% | 91.696% | 48.44 MB | 60fb3ef2e705 |
weights/sparse_attention_p3p4p5_matched_e86.weights.h5 |
Sparse Stage Attention ($P_3/P_4/P_5$) β Bit-for-bit matched pair winner (e86) | 6.27M | 73.878% | 91.616% | 48.44 MB | 830d6b48c11f |
weights/plain_b0_matched_control_e87.weights.h5 |
Plain EfficientNet-B0 β Bit-for-bit matched control (e87) | 5.33M | 72.924% | 91.030% | 41.13 MB | f8f0e061f947 |
weights/stage6_alternating_abab_1ep.weights.h5 |
Stage-6 Alternating Weight Sharing ($[B_1, A, B, A]$) β 1-ep continuation | 4.75M ($-10.9%$) | 70.916% | 89.940% | 36.67 MB | df14469a83e9 |
weights/stages4to6_single_template_1ep.weights.h5 |
Stages 4β6 Single-Template Sharing ($[B_1, A, A, A]$) β 1-ep continuation | 3.86M ($-27.6%$) | 66.104% | 86.992% | 29.84 MB | c3a2d4f5100b |
Repository Structure
.
βββ README.md
βββ SHA256SUMS.txt # Exact SHA-256 checksums for all weight files
βββ artifacts_manifest.json # Machine-readable metadata & checksum ledger
βββ code/
β βββ models.py # Plain EfficientNet-B0/B1/B2 ReLU6 model builder
β βββ attention_conv_model.py # Sparse Stage Attention (P3/P4/P5) model builder
β βββ shared_weight_efficientnet.py # Recurrent Weight Sharing ([B1,A,B,A] & [B1,A,A,A]) builder
βββ weights/
βββ sparse_attention_p3p4p5_standalone_e88.weights.h5
βββ sparse_attention_p3p4p5_matched_e86.weights.h5
βββ plain_b0_matched_control_e87.weights.h5
βββ stage6_alternating_abab_1ep.weights.h5
βββ stages4to6_single_template_1ep.weights.h5
Quickstart: Loading the Models in TensorFlow / Keras
from code.attention_conv_model import build_attention_conv_efficientnet
from code.models import build_efficientnet
from code.shared_weight_efficientnet import build_shared_stage6_pair_efficientnet_b0
# 1. Load Sparse Stage Attention (P3/P4/P5) β 74.02% Top-1 (standalone) or 73.88% (matched)
attn_model = build_attention_conv_efficientnet(variant="b0", num_classes=1000)
attn_model.load_weights("weights/sparse_attention_p3p4p5_standalone_e88.weights.h5")
# 2. Load Bit-for-Bit Matched Plain-B0 Control β 72.92% Top-1
ctrl_model = build_efficientnet(variant="b0", num_classes=1000)
ctrl_model.load_weights("weights/plain_b0_matched_control_e87.weights.h5")
# 3. Load Stage-6 Alternating Weight Sharing ([B1, A, B, A], -10.9% params) β 70.92% Top-1
abab_model = build_shared_stage6_pair_efficientnet_b0(num_classes=1000, num_stage6_blocks=4)
abab_model.load_weights("weights/stage6_alternating_abab_1ep.weights.h5")
Verify file integrity at any time with:
sha256sum -c SHA256SUMS.txt
Citation
@article{schroeder2026sparse,
title = {Where Global Context and Weight Sharing Pay Off in Compact Vision Backbones},
author = {Schroeder, Michael},
year = {2026}
}
- Downloads last month
- -
Dataset used to train meatloafer/efficientnet-b0-stage-attention
Evaluation results
- Top-1 Accuracy (Standalone Sparse Stage Attention P3/P4/P5, e88) on ImageNet-1k (50,000-image validation split)validation set self-reported74.020
- Top-1 Accuracy (Bit-for-Bit Matched Sparse Stage Attention P3/P4/P5, e86) on ImageNet-1k (50,000-image validation split)validation set self-reported73.878
- Top-1 Accuracy (Bit-for-Bit Matched Plain-B0 Control, e87) on ImageNet-1k (50,000-image validation split)validation set self-reported72.924
- Top-1 Accuracy (Stage-6 Alternating Weight Sharing [B1, A, B, A], 1-Ep Continuation) on ImageNet-1k (50,000-image validation split)validation set self-reported70.916