RunningTensor on ImageNet-1k, Tiny scale
Weights for every row of the ImageNet table: a RunningTensor vision backbone and the three published linear-time baselines, all trained under one matched recipe.
| Directory | Params | Final top-1 | Top-5 | Val loss |
|---|---|---|---|---|
rt-cumulative-gate |
7,081,096 | 76.10 | 92.96 | 1.0876 |
vim |
7,148,008 | 75.89 | 92.78 | 1.0790 |
gated-deltanet |
7,168,192 | 75.79 | 92.72 | 1.1028 |
vil-longhorn |
6,853,096 | 75.64 | 92.84 | 1.0839 |
rt-as-published |
7,081,096 | 75.38 | 92.60 | 1.1238 |
rt-legacy-wrapper |
7,144,060 | 71.10 | 89.95 | 1.3235 |
rt-conconlyc |
5,049,004 | 67.28 | 88.26 | 1.5078 |
ImageNet-1k validation split, 50,000 images. ViT-Tiny shell at width 192, global batch 1024, peak LR 1e-3 cosine to 1e-5, weight decay 0.1, drop_path 0, 1,244,160 images per epoch, 300 epochs.
rt-cumulative-gate and rt-as-published differ in one thing: whether the forget gate's g is
summed along the sequence. Summing it, which is what following GLA gives, is worth +0.72 top-1 and
is what puts the architecture ahead of all three baselines.
Files
Each directory holds final.pth and log.txt. The checkpoint carries model and model_ema
state dicts plus epoch and args; optimizer state was dropped, so these evaluate and fine-tune
but do not resume.
import torch
ck = torch.load("rt-cumulative-gate/final.pth", map_location="cpu", weights_only=False)
model.load_state_dict(ck["model"]) # or ck["model_ema"]
log.txt is one JSON object per epoch with train loss, validation loss, top-1, top-5 and the
learning rate.
Evaluating
Code, the notes string for each row, and the environment are in the vision_imagenet/ directory of
the repository this came from. For the top row:
export STMVIT_NOTES="depth:2_width:128_parallelpebbles_nopebbles_pffnstyle:full_delta:stm_stmgatetype:logsigmoid_stmtwobetas2_stmgouttime_stmgateclamp:-0.31_stmdk:12_stmdu:12_stmdv:48_stmfla_stmvitgcumsum_stmvitbidir:sum_tsarch:rtfla4_v42"
torchrun --nproc_per_node=1 vit/main.py --model stmlm_tiny_patch16_224 \
--data-set IMNET_PARQUET --data-path $DATA --batch-size 128 --num_workers 0 \
--groups_per_chunk 8 --eval --resume rt-cumulative-gate/final.pth
It prints the raw model first, then the EMA weights.