Mitchins/anime-style-classifier-v6

Anime Style Classifier v6

An EfficientNet-B2 model that classifies anime frames into 5 production style categories, expanding on v5 by splitting the painterly class into distinct pencil-sketch and painterly substyles. It can also produce 1,408-dimensional pooled feature embeddings for measuring visual similarity between frames. These embeddings are useful for style-oriented search, but are not guaranteed to be independent of image content.

Model Details

Architecture

EfficientNet-B2 (timm)

Parameters

7.7M

Input

440 × 440 RGB

Embedding dim

1,408 (global-average-pooled final feature map)

Classes

5 (flat, modern, retro, stylised_painterly, stylised_pencil)

Format

SafeTensors

File size

30 MB

Training data

97,444 images across 5 classes

Training method

Supervised ASL (Asymmetric Softmax) with batch-balanced sampling

Classes

Style	Description	Example Series
flat	Minimal shading, solid color blocks, low gradients	Ping Pong The Animation, Kaiba, The Heike Story
modern	High-detail digital pipeline, clean gradients, AAA production	Demon Slayer, Violet Evergarden, Attack on Titan
retro	Cel-era aesthetic, grain, limited dynamic range	Cowboy Bebop, Neon Genesis Evangelion, Robotech
stylised_painterly	Watercolor/oil feel, visible brush texture, organic transitions	Mushi-Shi, Your Lie in April, Land of the Lustrous
stylised_pencil	Sketch/pencil line art, loose strokes, minimalist shading	Kill la Kill, Carole & Tuesday, Penguindrum

Style Examples

flat	modern

retro	stylised_painterly

stylised_pencil

Performance

OOD Evaluation (459 held-out images, never seen during training)

Overall accuracy: 96.30% (442/459)

Class	Recall	Count	Errors
flat	98.6%	73/74	1
modern	99.0%	204/206	2
retro	94.0%	94/100	6
stylised_painterly	90.9%	40/44	4
stylised_pencil	88.6%	31/35	4

Confusion Matrix

| → flat | → modern | → retro | → painterly | → pencil ---|---|---|---|---|--- flat (74) | 73 | 0 | 1 | 0 | 0 modern (206) | 2 | 204 | 0 | 0 | 0 retro (100) | 1 | 5 | 94 | 0 | 0 painterly (44) | 1 | 2 | 0 | 40 | 1 pencil (35) | 2 | 0 | 1 | 1 | 31

Error Analysis (17 total)

All errors are borderline cases. Click any thumbnail to view full size.

Image	True	Predicted	Confidence	Notes
	flat	retro	51.8%	Cel-shaded chibi art resembles vintage style
	modern	flat	70.6%	Minimalist scene with almost no shading
	retro	modern	94.3%	High-confidence error — clean remaster
	retro	modern	88.5%	Muted palette but sharp compositing
	stylised_painterly	flat	92.0%	AI-gen with minimal texture; reads as flat
	stylised_painterly	stylised_pencil	96.0%	Ink-wash with strong pencil line dominance
	stylised_painterly	modern	55.8%	Synthetic frame; clean digital finish undermines painterly read
	stylised_pencil	retro	49.8%	Low-confidence; grainy scan resembles cel-era
	stylised_pencil	flat	77.6%	Clean lines with minimal visible stroke texture
	stylised_pencil	stylised_painterly	56.3%	Pastel wash over sketch — genuinely transitional

Comparison to V5 (4-class Baseline)

Metric	V5 (4-class)	V6 (5-class)	Notes
Overall OOD	98.41%	96.30%	−2.11pp (acceptable tradeoff for added granularity)
Flat	100.0%	98.6%	−1.4pp
Modern	98.5%	99.0%	+0.5pp ✓
Retro	99.0%	94.0%	−5pp (hard class, cel-era mastered series)
Painterly (V5)	94.0%	90.9% / 88.6%	V6 splits into painterly (90.9%) + pencil (88.6%)
Training Data	19,611	97,444	+397% (larger dataset, same architecture)
Architecture	Same	Same	EfficientNet-B2 @ 440px
Loss Function	Cross-entropy + merge	ASL (Asymmetric Softmax)	ASL delivers +8pp vs CE baseline

Why the Tradeoff?

V5's high accuracy (98.41% OOD) came from a small, highly-curated 19.6k image dataset and class-balanced sampling. V6 training expanded to 97.4k images across 5 classes to separate pencil and painterly:

• Data imbalance: Retro has nearly 15× more training samples than flat (dataset-driven split, not architecture-driven)
• Annotation subjectivity: The pencil class has 35 examples in the reported OOD benchmark (small sample size)
• Boundary ambiguity: Pencil/painterly blend on 2-3% of training images (subjective human labels)

The 2.11pp OOD drop is acceptable given the added classification granularity. Pencil is the weakest class at 88.6%; the other reported class recalls range from 90.9% to 99.0%.

Shared Class Performance (V5 v6 split analysis)

For direct comparison, merging V6's painterly + pencil back into single "painterly" class:

• V5 painterly: 94.0% (47/50 OOD)
• V6 painterly+pencil merged: 90.6% (71/79 OOD)
• Delta: −3.4pp (expected from increased training distribution mismatch)

Usage

Installation

pip install torch timm torchvision pillow

Classification

import torch
from safetensors.torch import load_file
import timm
from PIL import Image
from torchvision import transforms

# Load model
model = timm.create_model('efficientnet_b2', num_classes=5, pretrained=False)
state_dict = load_file('model.safetensors')
model.load_state_dict(state_dict)
model.eval()

# Preprocessing
transform = transforms.Compose([
    transforms.Resize((440, 440)),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
])

# Predict single image
image = Image.open('anime_frame.jpg').convert('RGB')
x = transform(image).unsqueeze(0)

with torch.no_grad():
    logits = model(x)
    probs = torch.softmax(logits, dim=1)
    pred_idx = logits.argmax(-1).item()
    confidence = probs[0, pred_idx].item()

classes = ['flat', 'modern', 'retro', 'stylised_painterly', 'stylised_pencil']
print(f'{classes[pred_idx]}: {confidence:.1%}')
# Output: modern: 96.5%

Batch Classification with Confidence

from torch.utils.data import DataLoader
from torchvision.datasets import ImageFolder
import pandas as pd

# Load images from folder
dataset = ImageFolder('anime_frames/', transform=transform)
loader = DataLoader(dataset, batch_size=32)

results = []
with torch.no_grad():
    for images, _ in loader:
        logits = model(images)
        probs = torch.softmax(logits, dim=1)
        preds = logits.argmax(-1)
        confs = probs.gather(1, preds.unsqueeze(1)).squeeze()
        
        for pred, conf in zip(preds, confs):
            results.append({'style': classes[pred], 'confidence': conf.item()})

df = pd.DataFrame(results)
print(df.groupby('style')['confidence'].agg(['count', 'mean', 'min']))

Confidence Thresholding (Optional)

If you want to reject ambiguous classifications:

with torch.no_grad():
    logits = model(x)
    probs = torch.softmax(logits, dim=1)
    pred_idx = logits.argmax(-1).item()
    confidence = probs[0, pred_idx].item()
    
    if confidence < 0.85:
        print('Ambiguous — may want to review manually')
    else:
        print(f'{classes[pred_idx]}: {confidence:.1%} (confident)')

Based on OOD evaluation:

• ≥ 0.95 confidence: >99% accurate
• 0.85–0.95 confidence: ~97% accurate
• < 0.85 confidence: Review recommended, especially for pencil class

Use as Style Embedding Model

Beyond classification, this model produces 1,408-dimensional embeddings by global-average-pooling the final convolutional feature map. They may capture visual drawing characteristics such as line weight, shading technique, color palette, and compositing approach, but can also retain scene-content information.

This enables:

• Batch style search: Find all frames in a 1.5M screencap corpus with matching visual drawing style
• Series identification: Cluster frames by series/studio without using metadata
• Style interpolation: Measure how a show's visual style evolves over seasons
• Synthetic dataset balance: Find style-similar examples of underrepresented classes

Embedding Usage

import torch
from safetensors.torch import load_file
import timm
from PIL import Image
from torchvision import transforms

# Load model
model = timm.create_model('efficientnet_b2', num_classes=5, pretrained=False)
state_dict = load_file('model.safetensors')
model.load_state_dict(state_dict)
model.eval()

transform = transforms.Compose([
    transforms.Resize((440, 440)),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
])

def get_style_embedding(image_path):
    """Extract a 1,408-dim embedding from the final convolutional features."""
    img = Image.open(image_path).convert('RGB')
    x = transform(img).unsqueeze(0)
    with torch.no_grad():
        features = model.forward_features(x)          # [1, 1408, 14, 14]
        embedding = features.mean(dim=[-2, -1])       # [1, 1408] global avg pool
        embedding = embedding / embedding.norm(dim=1, keepdim=True)  # L2 normalize
    return embedding.squeeze(0).numpy()

# Compare two frames
emb_a = get_style_embedding('frame_a.jpg')
emb_b = get_style_embedding('frame_b.jpg')
similarity = emb_a @ emb_b  # cosine similarity (embeddings are L2-normalized)
print(f'Style similarity: {similarity:.4f}')
# > 0.8  = very similar style (likely same series/studio)
# 0.5-0.8 = same style family (e.g., both modern)
# 0.2-0.5 = different but adjacent styles (e.g., modern vs retro)
# < 0.2  = opposite ends of spectrum (e.g., flat vs modern)

Interpreting Embedding Distances

Cosine Similarity	Interpretation
> 0.8	Near-identical style (same series, same studio, same season)
0.5 – 0.8	Same style family (e.g., two modern AAA shows, or two retro productions)
0.2 – 0.5	Different style families but compatible (e.g., modern + painterly)
< 0.2	Opposite ends of style spectrum (e.g., flat vs modern digital)

The embedding captures how something is drawn — line weight, shading technique, color palette, compositing — not what is drawn. Scenes with completely different content (landscapes vs close-ups vs action sequences) from the same series will cluster together.

Training Details

Data Composition

Class	Images	Source	Notes
flat	3,012	Real anime + curated synthetic	Underrepresented, lowest in-distribution diversity
modern	18,533	TV series + films	Largest class, 50+ contemporary productions
retro	44,520	Pre-digital era series	4× flat, 1980–2004 distribution
stylised_painterly	20,605	Real frames + artistic sources	Watercolor, oil, textured overlays
stylised_pencil	10,774	Split from original painterly	Sketch art, line-dominant, minimalist
Total	97,444	Multiple sources	Split: 78,144 train / 9,744 val / 9,556 held-out OOD

Loss Function Innovation: Asymmetric Softmax (ASL)

Standard cross-entropy treats all mistakes equally. Anime style classification is imbalanced:

• In-distribution: Most training images are clean, well-lit frames with clear style markers (easy to classify)
• Out-of-distribution (OOD): Test images include ambiguous boundary cases, poor lighting, style blends (hard to classify)

Asymmetric Softmax solves this by:

1. Hard negatives (γ_neg = 4.0): Penalize false positives heavily → sharp decision boundaries on negatives
2. Soft positives (γ_pos = 1.0): Keep positive examples less strict → model learns to be confident without overfitting

Formula: p_t = (1 - p)^γ_neg * CE for negatives; p_t^γ_pos * CE for positives

Impact: +8.04pp over the cross-entropy baseline on the separate 131-image OOD experiment (89.38% CE → 97.42% ASL)

This single loss choice was more impactful than:

• Merge experiments (best merge: 95.54% on that experiment, underperformed standalone ASL)
• Post-hoc fine-tuning (specialization attempted but minimal divergence from baseline)
• Different architectures (EfficientNet-B2 already optimal for this scale)

Training Pipeline

1. Base training: EfficientNet-B2 (ImageNet pretrained), ASL loss, batch-balanced sampling (upweight flat class 3×)
2. Optimizer: AdamW with 1e-3 initial LR, cosine annealing over 25 epochs
3. Validation: held-out OOD data; the model-card result above is reported on 459 images. The loss-comparison experiments below use a separate 131-image OOD split.
4. Best checkpoint: Epoch 9/25, saved based on OOD accuracy (not validation loss)

Why ASL over Alternative Losses?

Loss	Purpose	OOD Result	Notes
Cross-Entropy	Baseline	89.38%	Separate 131-image OOD experiment
ASL (γ_neg=4.0)	Hard negatives	97.42%	✓ Winner in the separate 131-image experiment
Focal Loss	Hard examples	95.54%	Same experiment; did not match ASL's specificity
Label Smoothing	Calibration	95.07%	Same experiment; hurt OOD performance on ambiguous cases

ASL's asymmetry perfectly matched the OOD challenge structure.

Limitations

Trained primarily on Japanese anime (TV series, films). May not generalize to:

• ❌ Western animation (different art direction traditions)
• ❌ Donghua (Chinese animation with distinct visual language)
• ❌ Manhwa (Korean webtoon styles)

Class-Specific Limitations

• ⚠️ Pencil class weak: 88.6% OOD accuracy (31/35). Limited training diversity and a subjective boundary with painterly. Its four errors were split across flat (2), retro (1), and painterly (1). Recommend a higher confidence threshold for pencil predictions.
• ⚠️ Painterly/pencil boundary: Subjective when styles blend (watercolor overlay on sketch lines, mixed-media). The reported confusion matrix contains 2 such cross-class errors out of 17 total errors.
• ⚠️ AI-generated art: Models trained on diffusion outputs (ComfyUI, Stable Diffusion) often misclassify between painterly and modern when they successfully mimic style.
• ⚠️ Flat class variability: Increased from 0 to 3 errors vs V5 due to larger training distribution mismatch.

What This Model is NOT

• ❌ Not a character design classifier — won't detect moe, chibi, bishounen, etc.
• ❌ Not a content classifier — sees how it's drawn, not what
• ❌ Not an era detector — a modern show using cel technique → retro (correct)
• ❌ Not a quality scorer — good and bad frames of the same style both classify identically
• ❌ Not an AI detector — AI art that successfully mimics a style gets classified by that style

Future Work (v6.1+)

Immediate Priorities (pencil class improvement)

The pencil class (88.6% OOD) is the main constraint on v6's overall performance. Paths forward:

1. Hard negative mining — Collect confident pencil misclassifications, especially near the painterly boundary. Retrain on these hard examples (20-50 images) to tighten the boundary.

2. Curriculum learning — Instead of fixed class weights, progressively increase pencil focus over epochs (0.5× → 1.0× → 3.0×), allowing the model to develop richer representations.

3. Synthetic pencil augmentation — Generate pencil-style images via style transfer or CLIP-guided diffusion to expand diversity.

4. Ensemble with V5 — Combine V6's 5-class precision with V5's 4-class robustness via voting or weighted logits to hedge pencil weakness.

Medium-term Improvements

1. Contrastive learning — Explicitly push pencil and painterly embeddings apart using contrastive loss on borderline examples.

2. OOD-aware training — If oracle OOD labels are available during training, use them in loss function to optimize for OOD rather than validation.

3. Multi-scale inference — Ensemble predictions at 440px and 512px (as V5 did) to capture different receptive fields.

Research Directions

1. Causal analysis — Which image features most influence pencil misclassifications? (Line density? Stroke boldness? Color saturation?)

2. Style interpolation — Measure how visual style evolves over a series' runtime using embeddings.

3. Synthetic dataset balancing — Use style embeddings to find and clone style-similar images for underrepresented classes (flat, pencil).

Citation

@misc{anime-style-classifier-v6,
  title={Anime Style Classifier V6},
  year={2026},
  publisher={HuggingFace},
  note={EfficientNet-B2 (7.7M) fine-tuned with Asymmetric Softmax loss for 5-class anime production style classification}
}

---

Model Card Summary

Metric	Value
Model	model.safetensors
Status	Production-ready
OOD Accuracy	96.30% (442/459 held-out images)
Weakest Class	stylised_pencil 88.6% (31/35 OOD)
File Size	30 MB
Parameters	7.7M
Input	440×440 RGB
Classes	5 (flat, modern, retro, stylised_painterly, stylised_pencil)
Loss Function	Asymmetric Softmax (γ_neg=4.0, γ_pos=1.0)
Training Data	97,444 images
Checkpoint Date	March 23, 2026
Comparison to V5	+1 class, +397% training data, −2.11pp OOD (acceptable tradeoff)

Recommended Use:

• Style classification of anime screencaps, scenes, frames
• Style embeddings for similarity search and clustering
• Dataset curation and filtering
• NOT for content classification or quality scoring

---

For questions, comparisons, or examples, see the V5 model card for additional context on the v5→v6 evolution.