OCT Retinal Classification · SCET · 2026

See inside
the machine
mind

Microscopic-level AI analysis of OCT B-scans — real feature maps, neural journey visualization, Mamba state dynamics, and clinical-grade explainability.

99.90%
Test Accuracy
96.68%
Val Accuracy
1.0000
AUC-ROC
101.2M
Parameters
0.9986
Cohen's κ
Read the Paper → Live Demo
Scroll
IEEE Conference Paper · 2026

RetViM: Sequential Hybrid Vision Transformer with MedMamba for Retinal Disease Classification

Khamir Desai, Mayuri A. Mehta, Sree Saicharan Vadapalli  ·  Sarvajanik College of Engineering and Technology, Surat, India

Early detection of retinal diseases such as Diabetic Macular Edema (DME), Choroidal Neovascularization (CNV) and Drusen is crucial for preventing irreversible vision loss. This paper proposes RetViM, a novel sequential hybrid method combining Vision Transformers (ViT) and Modified MedMamba for retinal disease classification using OCT B-scan images. RetViM achieves 99.90% accuracy, 99.90% precision, 99.90% recall, 99.97% specificity, 99.90% F1-score, AUC-ROC of 1.00 on the test set and 0.9986 Cohen's kappa — with only 1 misclassification out of 968 test images.
Deep Learning Vision Transformer MedMamba State Space Model Optical Coherence Tomography Retinal Disease
§ 1 — Motivation & Problem Statement

Why Retinal Disease
Classification Matters

"The retina is the only part of the central nervous system directly visible to an ophthalmologist. It is also where preventable blindness begins — and where AI can intervene before the window closes."

Clinical Burden
285 Million Visually Impaired
The WHO estimates 285 million people suffer visual impairment, with 39 million blind. Retinal diseases — including DME, CNV (wet AMD) and Drusen (dry AMD) — account for a major fraction of preventable blindness.
Diagnostic Gap
Specialists Cannot Scale
There is a critical global shortage of retinal specialists. Manual OCT interpretation is time-consuming, subjective, and costly. Rural and developing populations have near-zero access to specialist care.
AI Opportunity
OCT: Gold-Standard Imaging
Optical Coherence Tomography (OCT) is the gold-standard, non-invasive cross-sectional scan of the retina. It reveals layer-by-layer microstructure at micrometer resolution — making it ideal for AI analysis.
The Four Classes
CNV — Choroidal Neovascularization
Abnormal blood vessels grow beneath the retina, leaking fluid and causing rapid central vision loss. Associated with wet AMD. Requires urgent anti-VEGF treatment.
DME — Diabetic Macular Edema
Fluid accumulation in the macula caused by diabetic retinopathy. Cystoid spaces form in inner retinal layers. Leading cause of vision loss in working-age adults.
DRUSEN — Dry AMD
Sub-RPE lipid deposits (drusen) cause progressive photoreceptor atrophy. Intermediate stage of dry AMD. Requires monitoring; 18–30% progress to advanced AMD.
NORMAL — Healthy Retina
All nine retinal layers intact with normal reflectivity. Foveal pit preserved. IS/OS junction continuous. Correct ruling-out is equally important to avoid unnecessary treatment.
Research Gap This Paper Addresses
✗ Existing CNNs lack global context
ConvNets capture local patterns but miss long-range inter-layer relationships critical for retinal pathology.
✗ Pure ViTs are data-hungry
Vision Transformers need massive datasets. Medical imaging datasets are inherently small and imbalanced.
✗ Black-box predictions lack clinical trust
Clinical adoption requires explainability. Doctors need to understand why a prediction was made.
✓ RetViM: Hybrid Sequential Architecture
ViT-B/16 for global context + MedMamba for local selective state-space modeling + 6 XAI methods for clinical explainability.
Architecture

Sequential Hybrid
ViT-MedMamba

ViT-B/16 captures global long-range dependencies. Modified MedMamba refines with local Conv + selective State-Space modeling.

OCT B-Scan 224×224×3 ImageNet Norm Patch Embed 16×16 Conv2d 196 + [CLS] 768-dim ViT Blocks 1–6 FROZEN · ImageNet-21k MHSA FFN Low/mid-level features ViT Blocks 7–12 TRAINABLE · OCT fine-tuned MHSA FFN High-level OCT features Modified MedMamba · 2 Blocks Conv Branch DW 7×7/5×5/1×1 Local texture SSM Branch SS2D + 4-dir scan Global sequence Concat → 2-layer MLP + DropPath residual Multi-Scale Pool CLS token Avg pool Max pool Attn pool 4×768 → 768 MLP Classifier 768→512→256→4 GELU + Dropout Softmax → 4 CNV DME DRS NRM

Fig 1 — RetViM Architecture: OCT B-scan → ViT-B/16 (6 frozen + 6 trainable blocks) → 2 Modified MedMamba blocks (Conv ∥ SSM → MLP fusion) → Multi-scale pooling → 4-class softmax

Sequential Hybrid ViT-MedMamba Architecture
Architecture

Generated architecture visualization · 101.2M params · 12 ViT blocks (6 frozen) + 2 SS-Conv-SSM blocks · Multi-scale pooling fusion

§ 2 — Dataset

Kermany OCT 2017
84,484 Labeled Scans

One of the largest publicly available retinal OCT datasets. Curated by Daniel S. Kermany et al. at UC San Diego, validated by board-certified retinal specialists.

84,484
Training Images
4
Disease Classes
968
Test Images
242×4
Balanced test split
Split Distribution
Train
84,484
Val
32
Test
968
Test set: 242 images per class (balanced). Completely held-out — never seen during training or validation. This strict split prevents any data leakage.
Training Class Distribution
CNV
37,205
DME
11,348
DRUSEN
8,616
NORMAL
26,315
Class imbalance addressed via weighted CrossEntropyLoss during training.
Dataset Properties
Imaging ModalityOCT B-scan
Input Resolution224 × 224 px
Color Channels3 (RGB converted)
NormalizationImageNet μ/σ
Patch Size16 × 16 → 196 tokens
Label ValidationBoard-certified MDs
SourceMendeley Data
Augmentation (training only): Random horizontal flip, random rotation ±15°, color jitter (brightness 0.1, contrast 0.1), random resized crop (scale 0.9–1.0).
§ 3 — Methodology

How RetViM Works:
Step-by-Step Pipeline

A sequential hybrid architecture that first extracts global context via a frozen + fine-tuned ViT, then refines with selective state-space modeling via MedMamba blocks.

101.2M
Total Parameters
58.1M
Trainable
12 + 2
ViT + Mamba Blocks
768
Embedding Dim
O(N)
SSM Complexity
Pipeline Walkthrough
1

Input Preprocessing

OCT B-scan resized to 224×224, converted to RGB (3-channel), normalized with ImageNet statistics (μ=[0.485,0.456,0.406], σ=[0.229,0.224,0.225]) so pretrained weights are properly calibrated.

2

Patch Embedding (ViT-B/16)

Image split into 196 non-overlapping 16×16 patches. Each patch linearly projected to 768-dim embedding. A learnable [CLS] token prepended. Positional embeddings added.

3

Frozen ViT Blocks L1–L6

First 6 transformer encoder blocks kept frozen from ImageNet pretraining. These extract generalizable low-level features (edges, textures, spatial structure) without overfitting to the small medical dataset.

4

Fine-Tuned ViT Blocks L7–L12

Last 6 transformer blocks fine-tuned end-to-end. Multi-head self-attention (12 heads, 64-dim/head) captures global inter-patch relationships — critical for detecting diffuse pathologies like DME.

5

MedMamba SS-Conv-SSM Blocks × 2

Two sequential Modified MedMamba blocks refine the token sequence. Each block runs a parallel Conv branch (depthwise conv for local texture) and SSM branch (selective state-space scan for sequential dependencies), fused via MLP.

6

Multi-Scale Pooling & Classification

CLS token + mean-pooled patch tokens → dual-stream feature fusion → LayerNorm → 4-class linear head with softmax. Final prediction: argmax over [CNV, DME, DRUSEN, NORMAL].

Training Configuration
Optimizer
AdamW (β₁=0.9, β₂=0.999, ε=1e-8)
Learning Rate
2e-4 (frozen layers); 1e-5 (ViT fine-tune)
LR Schedule
CosineAnnealingLR (T_max=20, η_min=1e-7)
Weight Decay
1e-2 (AdamW regularization)
Loss Function
Weighted CrossEntropyLoss (imbalance correction)
Batch Size
32 (gradient accumulation × 2)
Epochs
20 (best at epoch 19: 96.68% val acc)
Precision
BF16 (Brain Float 16) mixed precision
Hardware
NVIDIA A100 80GB SXM4
Framework
PyTorch 2.x + PyTorch Lightning
Pretrained
ViT-B/16 — ImageNet-21k → ImageNet-1k
Why This Architecture?
Global + Local Synergy
ViT's self-attention sees the whole retina at once. MedMamba's conv branch captures fine-grained layer boundaries. Together they dominate at 99.90%.
Transfer Learning Efficiency
Freezing 6 ViT blocks prevents overfitting on 84K images. Only 58.1M of 101.2M parameters are trainable.
O(N) State-Space Complexity
MedMamba SSM processes sequences in O(N) vs O(N²) for full attention — enabling efficient modeling of long token sequences.
§ 4 — Experimental Results · Kermany OCT 2017

Benchmark
Performance

1
misclassification
out of 968 test images
RetViM misclassified exactly 1 image across a completely held-out test set of 968 scans — 242 per class. This corresponds to 99.90% accuracy, AUC-ROC 1.0000, and Cohen's κ = 0.9986 — a near-perfect agreement with board-certified specialist labels.
Table VIII — Full-Scale Model Comparison
Model Acc % Prec % Recall % Spec % F1 % AUC-ROC Kappa Errors
ViT-Base (Baseline) 99.38 99.38 99.38 99.79 99.39 1.0000 0.9917 6
Hybrid ViT-MedMamba v1 99.07 99.10 99.07 99.69 99.07 0.9999 0.9876 9
RetViM (Proposed) ★ 99.90 99.90 99.90 99.97 99.90 1.0000 0.9986 1
Table IX — Per-Class Performance of RetViM
Class Precision Recall Specificity F1 AUC Support
CNV 1.0000 1.0000 1.0000 1.0000 1.0000 242
DME 1.0000 1.0000 1.0000 1.0000 1.0000 242
DRUSEN 0.9959 1.0000 0.9986 0.9979 1.0000 242
NORMAL 1.0000 0.9959 1.0000 0.9979 1.0000 242
Macro Avg 0.9990 0.9990 0.9997 0.9990 1.0000 968
Confusion Matrix — 968 Test Images
Misclassification Analysis
1 Error out of 968 Images
Sample #862: NORMAL misclassified as DRUSEN
Decision margin: 1.61%
Why this is clinically acceptable:
1. Decision margin is only 1.61% — genuine ambiguity
2. Direction is conservative (false DRUSEN) — prompts monitoring rather than dismissal
3. The image exhibits subtle RPE irregularities creating real ambiguity
100%
CNV & DME Recall
99.90%
Overall Accuracy
0.9986
Cohen's Kappa
Table VI — Ablation Study (16,102 training images)
Model Accuracy % F1 % AUC-ROC Epochs Key Finding
ViT-Small 98.97 98.97 0.9999 50 Self-attention highly effective for OCT
Hybrid (4 blk) 98.55 98.56 0.9998 80 Over-regularization from 4 blocks
MedMamba 97.11 97.11 0.9995 150 Slow convergence, needs 3× epochs
Table IV — Kermany OCT 2017 Dataset Distribution
CNV
33,509
30,258 train · 3,009 val · 242 test
DME
10,213
9,192 train · 779 val · 242 test
DRUSEN
7,768
6,992 train · 534 val · 242 test
NORMAL
33,006
28,731 train · 4,033 val · 242 test
Total: 84,496 OCT B-scan images · 75,173 train · 8,355 val · 968 test (balanced)
§ 5 — Explainability · Trustworthy AI

Six XAI Methods for
Clinical Transparency

A black-box 99.9% accurate model is not enough for clinical use. RetViM is paired with six complementary explainability methods — each answering a different question about the model's decision.

A 99.9% accurate model that a clinician cannot interrogate is not a clinical tool — it is a liability. RetViM ships six independent explanations with every prediction.

01 · GRADCAM
GradCAM
Gradients of the target class flow into the final attention layer, weighting each spatial feature map. Produces a coarse heatmap highlighting the most discriminative retinal regions — fast enough for clinical screening.
"Which spatial regions most strongly activated this prediction?"
BackpropagationSpatial map~12ms
02 · GRADCAM++
GradCAM++
Second-order derivatives allow finer localization of multiple distinct pathological instances. GradCAM++ down-weights diffuse background gradients and elevates sharp, high-confidence activation peaks.
"Where are the sharpest, highest-confidence hotspots?"
2nd-orderMulti-instance
03 · ATTENTION ROLLOUT
Attention Rollout
Multiplies all 12 transformer attention matrices together (accounting for residual connections) to trace exactly how information from each input patch flows into the final CLS token — a direct read of ViT's global attention.
"What does the full 12-layer attention graph attend to?"
Transformer-native12 layers
04 · OCCLUSION
Occlusion Sensitivity
A sliding grey patch systematically occludes every region of the OCT scan. The drop in prediction confidence at each location reveals which pixels causally matter — no gradient required. Fully model-agnostic.
"Which region, if hidden, most confuses the model?"
CausalModel-agnostic
05 · INTEGRATED GRADIENTS
Integrated Gradients
Integrates gradients along the straight interpolation path from a black baseline to the input, satisfying the completeness axiom: attributions sum exactly to the output score difference. The most theoretically principled pixel-level explanation.
"What is each pixel's mathematically fair contribution?"
Axiomatic25-step path
06 · RISE
RISE
40 random binary masks are applied to the input; each mask's prediction score becomes its weight. The weighted average of all masks is a statistical saliency map robust to individual noise — ideal for validating other methods.
"Which pixels statistically correlate with correct predictions?"
Stochastic40 masks
Multi-Method Clinical Consensus
When GradCAM, Attention Rollout, and Integrated Gradients all independently highlight the same anatomical region — for instance, the RPE/Bruch's membrane interface in a CNV prediction — a clinician can trust the model's decision because three methods using fundamentally different mathematical principles arrived at the same answer. This multi-method consensus is a core contribution of this paper.
§ 6 — Neural Journey · Feature Transformation · PyTorch

The Neural
Journey

Watch how a retinal OCT scan transforms through every layer of the hybrid ViT-Mamba pipeline. Row 1: smoothed feature heatmaps · Row 2: raw single-channel activations · Row 3: attention overlays (where the network looks). Compare Demo vs Real model weights side-by-side.

Demo vs Real
Model Architecture
12 ViT
Transformer Blocks
4 vs 2
Mamba Blocks
384 vs 768
Embedding Dim
96.68%
Real Model Val Acc
Demo · RetViMNet dim=384 · 12ViT + 4Mamba
Demo Neural Journey
Demo Weights

Real · ImprovedMedMamba dim=768 · 12ViT + 2Mamba
Real Neural Journey
Real Weights · 96.68%

Rows:
Feature Heatmaps (smoothed spatial mean)
Raw Activations (high-variance channel)
Attention Overlays (jet blend + contours)
ViT blocks
Mamba blocks
Live Analysis

Two-mode
Intelligence

XAI Suite · CKA Matrix · Mamba Engine · Layer Inspector · Feature Space · Performance
OCT B-Scan InputSelect image
Drop OCT B-scan here
PNG, JPG · 224×224+
Sample scans:
Select a sample · click Analyze
Researcher: 6 panels · Doctor: 4 panels
RetViM Neural Engine v4.0
CPU
$ retvim --mode researcher --input scan.png
Initializing... 0%
Deep Analysis · Real Model Outputs

Research
Visualizations

Every figure generated from the actual trained RetViM model on real Kermany OCT data.

DEMO · RetViMNet 384d · 4×Mamba
Demo Journey
Demo Weights

RetViMNet · 384-dim · ViT-Small transfer + OCT fine-tune · 12 ViT + 4 Mamba blocks

REAL · ImprovedMedMamba 768d · 2×Mamba
Real Journey
Real · 96.68% val acc

ImprovedMedMamba · 768-dim · ViT-Base/16 + 2 MedMamba blocks · val_acc = 96.68%

Rows:
Feature Heatmaps
Raw Activations
Attention Overlays
ViT Layers
Mamba Layers
Feature Journey All 4 Classes
All 4 Classes · Feature Journey

Input → Patch Embed → ViT Blk 3/7/11 → Mamba Blk 0/1/2/3 · Each row = one disease class · ViT stages: cool tones · Mamba stages: warm inferno

Feature Journey v2
Feature Journey v2

Alternative feature evolution view · cleaner layout showing progression from raw pixel features to diagnostic representations

DME Feature Evolution
DME Feature Evolution

DME classification · attention focuses on INL/OPL cystoid spaces · Mamba blocks refine the central macular signal · attention progressively concentrates

Class Discrimination
How Network Distinguishes 4 Classes

PCA across all pipeline stages (Patch Embed → ViT Blk 3/11 → Mamba Blk 0/1/2/3) + t-SNE final layer + average final-layer activations per class · Classes become perfectly separable by Mamba stage

Disease-Specific Activation Patterns
Disease-Specific Activation Patterns

Row = disease class · Columns = ViT Blk 3, ViT Blk 11, Mamba 0, Mamba 2, Mamba 3 · Each uses class-specific color: CNV=red, DME=blue, Drusen=gold, Normal=green

Class Visual Signatures
Class Visual Signatures

Visual signatures of each disease — Original OCT → ViT early/late features → Mamba features → final GradCAM overlay · Each class has distinct network fingerprint

Inside Neural Blocks
Inside ViT Attention + Mamba State-Space

Top: ViT self-attention architecture (Q·Kᵀ/√d) + SS-Conv-SSM dual branch (h'=Ah+Bx, y=Ch+Dx) · Bottom rows: actual feature maps at Early ViT/Middle ViT/Late ViT/Early Mamba/Middle Mamba/Final Mamba

PCA Feature Space Across Layers
PCA Feature Space Across 8 Layers

8-panel PCA: Patch Embed → ViT Blk 3/7/11 → Mamba Blk 0/1/2/3 · Patch Embed: overlapping butterfly shape → ViT Blk 11: classes begin separating → Mamba: tight well-separated clusters · Var explained increases Blk-11 67.7% → Mamba 70.1%

Activation Heatmaps Across Classes
Activation Heatmaps — All 4 Classes

What the network sees: Original → Patch Embed → ViT Blk 7 → Mamba Blk 0 → Mamba Blk 3 · Color-coded by disease: CNV=red, DME=cyan, Drusen=gold, Normal=teal · Brighter = higher activation

Neural Network Feature Dictionary
Feature Dictionary — Maximally Activating Patterns

8×8 grid · Rows = Patch Embed, ViT Blk 0/3/7/11 (blue), Mamba Blk 0/2/3 (red) · Columns = 8 channels · Each cell = synthetic input that maximally activates that channel · Early: color grids → Late: curved retinal structures

Hierarchical Feature Visualization
Hierarchical Feature Tree

Feature lineage across patch-embed → vit-3 → vit-7 → vit-11 → mamba-1 → mamba-3 · Lines show which features are preserved/transformed · Blue = ViT self-attention · Orange = Mamba state-space

DeepDream — Amplifying Network Perception
DeepDream — What Network Amplifies

Enhancing features the network detects in real OCT images · Original → ViT Block 5 (frozen, mid-level) → ViT Block 11 (trainable, semantic) → Combined · Network amplifies retinal layer boundaries and pathological textures

Benchmark · Kermany OCT 2017 · 968 test images

84,496 images trained.
99.90% test accuracy. 1 error in 968.

99.90%
Test Accuracy
967/968 images
96.68%
Val Accuracy
epoch 19 / 20
1.0000
AUC-ROC
4-class one-vs-rest
0.9986
Cohen's Kappa
near-perfect agreement
101.2M
Parameters
58.1M trainable
§ 7 — Citation & Copyright

Cite This
Paper

If you use RetViM in your research, please cite the following. The paper is accepted/published in IEEE conference proceedings.

bibtex — retvim2026.bib
@inproceedings{desai2026retvim, title = {RetViM: Sequential Hybrid Vision Transformer with MedMamba for Retinal Disease Classification}, author = {Desai, Khamir and Mehta, Mayuri A. and Vadapalli, Sree Saicharan}, booktitle = {IEEE Conference Proceedings}, year = {2026}, institution = {Sarvajanik College of Engineering and Technology, Surat, India}, note = {ViT-B/16 + 2 SS-Conv-SSM MedMamba blocks; 99.90% test accuracy on Kermany OCT 2017} }
IEEE Citation Format
K. Desai, M. A. Mehta, and S. S. Vadapalli, "RetViM: Sequential Hybrid Vision Transformer with MedMamba for Retinal Disease Classification," in IEEE Conference Proceedings, 2026.
IEEE Copyright Notice
© 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

This companion website is maintained by the authors for academic and educational purposes. All figures, diagrams, interactive demos, and code on this site are original author contributions. The abstract is reproduced with author permission consistent with IEEE author rights. For the published paper, please refer to IEEE Xplore.

Dataset: Kermany, D.S. et al. "Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning." Cell, 172(5), 1122–1131.e9, 2018. Accessed via Mendeley Data.