resnet-cifar10-68k

Model description

This model is an ultra‑compact ResNet (≈ 68,786 parameters) trained from scratch on the CIFAR-10 dataset. The architecture is a heavily width‑reduced ResNet20 (base width 8 instead of 16), giving only ~68k parameters. Training was performed on a consumer AMD Radeon RX 6600 GPU using PyTorch with ROCm support.

Author: sapbot from Romarchive

Intended uses

• Image classification on 10 everyday object categories.
• Demonstrating that extremely small networks can still achieve reasonable accuracy from scratch.

Training procedure

• Optimizer: SGD with momentum 0.9 and weight decay 1e‑4.
• Learning rate: 0.1, dropped by factor 0.1 at epochs 80 and 120 (MultiStepLR).
• Batch size: 128.
• Epochs: up to 160 (early stopping with patience 10).
• Data augmentation: random horizontal flip, random crop with 4‑pixel padding.
• Input normalization: mean (0.4914, 0.4822, 0.4465), std (0.2023, 0.1994, 0.2010).

Evaluation results

Metric	Value
Test accuracy (10k images)	82.29%
Test loss	0.5378
Full CIFAR-10 accuracy (60k images)	84.62%

Per‑class performance

                precision    recall  f1-score   support

  0 (airplane)     0.7599    0.8860    0.8181      1000
1 (automobile)     0.8670    0.9520    0.9075      1000
      2 (bird)     0.7869    0.7420    0.7638      1000
       3 (cat)     0.7237    0.6680    0.6947      1000
      4 (deer)     0.8202    0.8120    0.8161      1000
       5 (dog)     0.7429    0.7860    0.7638      1000
      6 (frog)     0.7777    0.9200    0.8429      1000
     7 (horse)     0.9160    0.7960    0.8518      1000
      8 (ship)     0.9198    0.8720    0.8953      1000
     9 (truck)     0.9672    0.7950    0.8727      1000

      accuracy                         0.8229     10000
     macro avg     0.8281    0.8229    0.8227     10000
  weighted avg     0.8281    0.8229    0.8227     10000

Confusion matrix (row = true label, col = predicted label)

True \ Pred	Pred 0	Pred 1	Pred 2	Pred 3	Pred 4	Pred 5	Pred 6	Pred 7	Pred 8	Pred 9
True 0	886	11	26	7	13	2	14	6	30	5
True 1	12	952	1	3	1	4	11	1	9	6
True 2	57	1	742	33	47	45	60	10	4	1
True 3	23	5	58	668	32	117	69	19	6	3
True 4	15	2	34	31	812	33	53	17	3	0
True 5	10	1	28	106	27	786	26	13	2	1
True 6	5	0	25	27	11	9	920	2	1	0
True 7	21	3	20	38	46	58	11	796	3	4
True 8	80	13	5	7	1	3	9	3	872	7
True 9	57	110	4	3	0	1	10	2	18	795

Model size

68,786 trainable parameters.

How to use

import torch
import torch.nn as nn
import torchvision.transforms as transforms
from PIL import Image

# --------------------------------------------------
# Ultra‑compact ResNet (exact architecture)
# --------------------------------------------------
def conv3x3(in_planes, out_planes, stride=1):
    return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride, padding=1, bias=False)

class BasicBlock(nn.Module):
    expansion = 1
    def __init__(self, in_planes, planes, stride=1):
        super(BasicBlock, self).__init__()
        self.conv1 = conv3x3(in_planes, planes, stride)
        self.bn1 = nn.BatchNorm2d(planes)
        self.relu = nn.ReLU(inplace=True)
        self.conv2 = conv3x3(planes, planes)
        self.bn2 = nn.BatchNorm2d(planes)
        self.shortcut = nn.Sequential()
        if stride != 1 or in_planes != self.expansion * planes:
            self.shortcut = nn.Sequential(
                nn.Conv2d(in_planes, self.expansion * planes, kernel_size=1, stride=stride, bias=False),
                nn.BatchNorm2d(self.expansion * planes)
            )

    def forward(self, x):
        out = self.relu(self.bn1(self.conv1(x)))
        out = self.bn2(self.conv2(out))
        out += self.shortcut(x)
        out = self.relu(out)
        return out

class TinyResNet(nn.Module):
    def __init__(self, block, num_blocks, base_width=8, num_classes=10):
        super(TinyResNet, self).__init__()
        self.in_planes = base_width
        self.conv1 = conv3x3(3, base_width)
        self.bn1 = nn.BatchNorm2d(base_width)
        self.relu = nn.ReLU(inplace=True)
        self.layer1 = self._make_layer(block, base_width, num_blocks[0], stride=1)
        self.layer2 = self._make_layer(block, base_width * 2, num_blocks[1], stride=2)
        self.layer3 = self._make_layer(block, base_width * 4, num_blocks[2], stride=2)
        self.avg_pool = nn.AdaptiveAvgPool2d((1, 1))
        self.fc = nn.Linear(base_width * 4 * block.expansion, num_classes)

    def _make_layer(self, block, planes, num_blocks, stride):
        strides = [stride] + [1] * (num_blocks - 1)
        layers = []
        for stride in strides:
            layers.append(block(self.in_planes, planes, stride))
            self.in_planes = planes * block.expansion
        return nn.Sequential(*layers)

    def forward(self, x):
        out = self.relu(self.bn1(self.conv1(x)))
        out = self.layer1(out)
        out = self.layer2(out)
        out = self.layer3(out)
        out = self.avg_pool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        return out

def ResNet64k():
    return TinyResNet(BasicBlock, [3, 3, 3], base_width=8)

# --------------------------------------------------
# Load the model weights
# --------------------------------------------------
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = ResNet64k().to(device)
model.load_state_dict(torch.load("best_resnet64k_cifar10.pth", map_location=device))
model.eval()

# --------------------------------------------------
# Preprocess an image (must be 32x32 RGB)
# --------------------------------------------------
transform = transforms.Compose([
    transforms.Resize((32, 32)),
    transforms.ToTensor(),
    transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
])

def predict(image_path):
    image = Image.open(image_path).convert('RGB')
    input_tensor = transform(image).unsqueeze(0).to(device)
    with torch.no_grad():
        output = model(input_tensor)
        _, predicted = output.max(1)
    classes = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
    return classes[predicted.item()]

# Example usage:
# print(predict("my_cat.jpg"))

Acknowledgements

• Original CIFAR‑10 dataset by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.
• PyTorch team for the framework and ROCm support.