README.md

---
license: mit
language:
- en
tags:
- banknote
- ocr
- nepali
- computer-vision
- object-detection
- image-classification
- yolo
- convnext
- swin
- rf-detr
library_name: pytorch
---

# Nepali Banknote Models

A collection of models for the **bankNotes-OCR** pipeline — a hierarchical 2-stage banknote retrieval system for Nepalese banknotes.

## Pipeline Overview

```
Query Image
  ├── Stage 0: ConvNeXt Binary Classifier  (banknote vs. random)
  ├── Stage 1a: YOLO Denomination Detector  (11 denominations)
  ├── Stage 1b: RFDETR + Swin Signature Classifier  (20 governors)
  └── Stage 2: DinoV2-Base patch embeddings + Qdrant vector search
```

---

## Model 1: ConvNeXt Tiny — Binary Classifier (`best_model.pth`)

**Stage 0:** Determines if an image is a banknote or a random image.

| | |
|---|---|
| **Architecture** | `convnext_tiny` (torchvision) |
| **Classes** | 2: random (0), banknote (1) |
| **Input size** | 224x224 RGB |
| **Normalization** | ImageNet mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225] |
| **Weight size** | ~319 MB |

```python
from huggingface_hub import hf_hub_download
import torch
from src.models.resnet import get_model

model_path = hf_hub_download("MANTRAIDEAS1/nepali-banknote-models", "best_model.pth")
model = get_model(model_name="convnext_tiny", num_classes=2, pretrained=False)
checkpoint = torch.load(model_path, map_location="cpu", weights_only=False)
model.load_state_dict(checkpoint["model_state_dict"])
model.eval()
```

---

## Model 2: YOLO — Denomination Detector (`best.pt`)

**Stage 1a:** Detects and classifies the denomination of a banknote.

| | |
|---|---|
| **Architecture** | YOLO (Ultralytics) |
| **Classes** (11) | 1, 2, 5, 10, 20, 25, 50, 100, 250, 500, 1000 |
| **Input size** | 640x640 RGB |
| **Weight size** | ~113 MB |

```python
from ultralytics import YOLO
from huggingface_hub import hf_hub_download

model_path = hf_hub_download("MANTRAIDEAS1/nepali-banknote-models", "best.pt")
model = YOLO(str(model_path))
results = model("image.jpg", conf=0.5, device="cpu")
```

---

## Model 3: RFDETR-Large — Signature Detector (`detector.pth`)

**Stage 1b:** Detects the governor signature region on a banknote.

| | |
|---|---|
| **Architecture** | RFDETRLarge (rfdetr library) |
| **Detection threshold** | 0.3 |
| **Weight size** | ~128 MB |

```python
from rfdetr import RFDETRLarge
from huggingface_hub import hf_hub_download

model_path = hf_hub_download("MANTRAIDEAS1/nepali-banknote-models", "detector.pth")
detector = RFDETRLarge.from_checkpoint(model_path, device="cpu")
```

---

## Model 4: Swin-Base — Signature Classifier (`classifier.pth`)

**Stage 1b:** Classifies a cropped signature region into one of 20 Nepali governors.

| | |
|---|---|
| **Architecture** | `swin_base_patch4_window7_224` (timm) |
| **Classes** | 20 Nepali governors |
| **Input size** | 224x224 RGB |
| **Normalization** | ImageNet mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225] |
| **Weight size** | ~331 MB |

### Governor Classes

```
Bharat_Raj_Pandey, Bhekh_Bahadur_Thapa, Bijaynath_Bhattarai,
Chiranjibi_Nepal, Dipendra_Purush_Dhakal, Ganesh_Bahadur_Thapa,
Hari_Shankar_Tripathi, Himalaya_SJB_Rana, Janak_Raj_Pandey,
Kalyan_Bikram_Adhikari, Krishna_Bahadur_Manandhar, Kul_Sekhar_Sharma,
Laxmi_Nath_Gautam, Maha_Prasad_Adhikari, Narendra_Raj_Pandey,
Pradhumna_Lal_Rajbhandari, Satyendra_Pyara_Shrestha,
Tilak_Bahadur_Rawal, Yadav_Prasad_Pant, Yubaraj_Khatiwada
```

```python
import timm
import torch
from huggingface_hub import hf_hub_download

model_path = hf_hub_download("MANTRAIDEAS1/nepali-banknote-models", "classifier.pth")
model = timm.create_model("swin_base_patch4_window7_224", pretrained=False, num_classes=20)
model.load_state_dict(torch.load(model_path, map_location="cpu"))
model.eval()
```

---

## Additional Model (not in this repo)

| Model | Source | Purpose |
|-------|--------|---------|
| **DinoV2-Base** | `facebook/dinov2-base` | Stage 2 — patch-level embeddings for visual similarity search |

## License

MIT