Squashing commit

Co-authored-by: jimmymendez <jimmymendez@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+weights/best.torchscript filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,331 @@
+---
+tags:
+- efficient-net
+- image-classification
+- document-analysis
+- visual-elements
+- cultural-heritage
+- digitized-books
+datasets:
+- institutional/institutional-books-hl-visual-elements
+pipeline_tag: image-classification
+model-index:
+- name: institutional-books-visual-elements-orientation
+  results:
+  - task:
+      type: image-classification
+      name: Visual Element Orientation Classification
+    metrics:
+    - type: accuracy
+      name: Overall accuracy
+      value: 0.9134
+      dataset:
+        type: institutional/institutional-books-hl-visual-elements
+        name: Visual elements crops (orientation test split)
+    - type: accuracy
+      name: upright accuracy
+      value: 0.9178
+    - type: accuracy
+      name: rotate_90_clockwise accuracy
+      value: 0.8976
+    - type: accuracy
+      name: rotate_180 accuracy
+      value: 0.9191
+    - type: accuracy
+      name: rotate_90_counterclockwise accuracy
+      value: 0.9191
+license: apache-2.0
+---
+
+# 📚 Institutional Books — Visual Elements Orientation Model (EfficientNetV2-M)
+
+A 4‑class image classification model that predicts the rotation correction needed to restore visual elements from digitized book page scans to upright orientation. This model operates on cropped regions (e.g., images, diagrams, ornaments) and is intended as a post-processing stage after visual-element detection.
+
+**More information:**
+- 📄 [Full analysis available in our technical report](TODO)
+- 🌐 [Website](https://institutional.org/institutional-books)
+
+**See also:**
+- ⚙️ [Pipeline](https://github.com/institutional/institutional-books-visual-elements-pipeline)
+- 🤖 [Detection model](https://huggingface.co/institutional/institutional-books-visual-elements-detection)
+- 🤖 [Classification model](https://huggingface.co/institutional/institutional-books-visual-elements-classification)
+
+The Institutional Data Initiative at Harvard Law School Library works with knowledge institutions—from libraries and museums to cultural groups and government agencies—to refine and publish their collections as data. [Reach out to collaborate on your collections](https://institutional.org/).
+
+---
+
+## Outline
+
+- [Model Description](#model-description)
+  - [Classes](#classes)
+- [Intended Use](#intended-use)
+- [Training](#training)
+  - [Dataset](#dataset)
+  - [Training Configuration](#training-configuration)
+  - [Data Augmentation (Train Only)](#data-augmentation-train-only)
+- [Evaluation](#evaluation)
+  - [Per-Epoch Training Summary](#per-epoch-training-summary)
+  - [Test Set Performance](#test-set-performance)
+- [Inference Configuration](#inference-configuration)
+- [Usage](#usage)
+  - [PyTorch Example](#pytorch-example)
+  - [Applying Corrections](#applying-corrections)
+- [Limitations](#limitations)
+- [Citation](#citation)
+
+---
+
+## Model Description
+
+- **Architecture:** EfficientNetV2-M
+- **Parameters:** 52,863,480
+- **Classes (corrections):** 4
+- **Input resolution (train/inference):** 480×480 px crops (from 512×512 resize)
+- **Framework:** PyTorch
+
+The model predicts the *inverse rotation* required to correct each crop back to upright. Input crops may be in any of four orientations; the model outputs one of four rotation labels.
+
+### Classes
+
+The labels are “correction actions” to make the image upright:
+
+| Class index | Label                          | Description                                           |
+|-------------|--------------------------------|-------------------------------------------------------|
+| 0           | **upright**                    | No rotation needed (already upright)                 |
+| 1           | **rotate_90_clockwise**        | Rotate 90° clockwise to correct                      |
+| 2           | **rotate_180**                 | Rotate 180° to correct                               |
+| 3           | **rotate_90_counterclockwise** | Rotate 90° counter-clockwise to correct              |
+
+Classes are mutually exclusive.
+
+## Intended Use
+
+This model classifies the orientation of **already-detected visual elements** from digitized book pages.
+
+**Primary use cases:**
+- Correcting rotations of cropped visual elements in digitization pipelines
+- Normalizing orientation before downstream tasks (captioning, OCR on musical scores, layout analysis)
+- Quality control on large-scale digitized collections (flagging mis-rotate elements)
+
+**Out of scope:**
+- General full-page orientation detection (expects tight crops of a single element)
+- Classification of semantic content (e.g., “image vs. music”)
+- Arbitrary-angle rotation beyond multiples of 90°
+
+## Training
+
+### Dataset
+
+Source data consists of **7,904** manually curated crops of visual elements from the [Institutional Books collection](https://huggingface.co/collections/institutional/institutional-books).
+
+Original (pre-synthetic) label distribution (estimated, by source orientation):
+
+- **upright:** 93.0%  
+- **rotate_90_clockwise:** 5.6%  
+- **rotate_90_counterclockwise:** 1.3%  
+- **rotate_180:** 0.03%  
+
+To avoid this extreme imbalance and to directly learn the correction operation:
+
+1. All images were first **manually corrected to upright**.
+2. Each upright image was **synthetically rotated** by 0°, 90°, 180°, and 270°.
+3. The target label is the **inverse rotation** needed to restore the upright orientation.
+
+Resulting synthetic orientation dataset:
+
+- **Total samples (with synthetic rotations):** 31,616
+- **Train samples:** 25,292  
+- **Val samples:** 3,160  
+- **Test samples:** 3,164  
+- **Split:** 0.8 / 0.1 / 0.1 (train / val / test)
+
+Each original crop contributes four synthetic samples (one per orientation), producing a **balanced label distribution** across the four classes in the synthetic set.
+
+### Training Configuration
+
+| Parameter              | Value                                  |
+|------------------------|----------------------------------------|
+| Backbone               | EfficientNetV2-M                       |
+| Classifier head        | Dropout(p=0.3) → Linear(1280, 4)       |
+| Image size (train)     | Resize(512×512) → RandomCrop(480×480) |
+| Image size (val/test)  | Resize(512×512) → CenterCrop(480×480) |
+| Batch size             | 32                                     |
+| Max epochs             | 20                                     |
+| Optimizer / LR         | Not specified (standard schedule)     |
+| Normalization          | ImageNet mean/std                      |
+| Hardware               | Single NVIDIA GH200 GPU                |
+| Total training time    | 58 min 13 sec (20 epochs)             |
+| Avg. per epoch         | ~2 min 55 sec                         |
+| Train samples/epoch    | 25,292 (~791 steps/epoch)             |
+| Throughput             | ~145 images/sec                       |
+
+### Data Augmentation (Train Only)
+
+Preprocessing:
+
+- `Resize(512×512)`
+- `RandomCrop(480×480)`
+- `ColorJitter(brightness=0.2, contrast=0.2, saturation=0.1)`
+
+Stochastic augmentations:
+
+| Augmentation        | Implementation                    | Probability |
+|---------------------|-----------------------------------|-------------|
+| Random auto-contrast| `RandomAutocontrast`             | p = 0.3     |
+| Random invert       | `RandomInvert`                   | p = 0.15    |
+| Random grayscale    | `RandomGrayscale`                | p = 0.2     |
+| Gaussian blur       | `GaussianBlur(k=5, σ=0.1–2.0)`   | p = 1.0     |
+| Random erasing      | `RandomErasing(scale=0.02–0.15)` | p = 0.2     |
+
+Validation/Test preprocessing:
+
+- `Resize(512×512)`
+- `CenterCrop(480×480)`
+- `Normalize` (ImageNet stats)
+
+## Evaluation
+
+### Per-Epoch Training Summary
+
+On the validation set, accuracy increases steadily and plateaus around **90%**, while validation loss begins to rise after approximately epoch 10, indicating **moderate overfitting**. Heavy augmentations successfully limit overfitting enough that the **held-out test set slightly outperforms validation**.
+
+Final epoch (20):
+
+- **Train accuracy:** 99.81%  
+- **Val accuracy:** 90.35%  
+- **Train loss:** 0.0058  
+- **Val loss:** 0.4753  
+
+### Test Set Performance
+
+On the held-out test set (3,164 samples; 791 per class):
+
+- **Overall accuracy:** 91.34% (2,890 / 3,164)
+
+Per-class accuracy:
+
+| Class                        | Accuracy | Correct / Total |
+|------------------------------|----------|-----------------|
+| **upright**                  | 91.78%   | 726 / 791       |
+| **rotate_90_clockwise**      | 89.76%   | 710 / 791       |
+| **rotate_180**               | 91.91%   | 727 / 791       |
+| **rotate_90_counterclockwise**  | 91.91% | 727 / 791   |
+
+Additional notes:
+
+- **Misclassifications:** 274 of the 3,164 test samples are misclassified (2,890 correct → 91.34% overall accuracy). These break down per class as 65 (`upright`), 81 (`rotate_90_clockwise`), 64 (`rotate_180`), and 64 (`rotate_90_counterclockwise`).
+- The 90° clockwise class is the most challenging, but still achieves close to 90% accuracy.
+
+## Inference Configuration
+
+Typical inference settings:
+
+| Parameter       | Value                             |
+|-----------------|-----------------------------------|
+| Image size      | 512×512 resize → 480×480 center crop |
+| Batch size      | 32 (tune for available GPU memory) |
+| Normalization   | ImageNet mean/std                 |
+| Output          | 4-way softmax over orientation labels |
+
+The top-1 prediction corresponds to the **rotation to apply** to make the crop upright.
+
+## Usage
+
+### PyTorch Example
+
+```python
+import torch
+import torch.nn as nn
+import torchvision.models as models
+from torchvision import transforms
+from huggingface_hub import hf_hub_download
+from PIL import Image
+
+# Download weights (the repo ships a state_dict at weights/weights.pth)
+model_path = hf_hub_download(
+    repo_id="institutional/institutional-books-visual-elements-orientation",
+    filename="weights/weights.pth",
+)
+
+# Build the architecture and load the state_dict
+model = models.efficientnet_v2_m(weights=None)
+num_features = model.classifier[1].in_features
+model.classifier = nn.Sequential(
+    nn.Dropout(p=0.3, inplace=True),
+    nn.Linear(num_features, 4),
+)
+state_dict = torch.load(model_path, map_location="cuda", weights_only=True)
+model.load_state_dict(state_dict)
+model.to("cuda")
+model.eval()
+
+# Preprocessing: match validation/test pipeline
+preprocess = transforms.Compose([
+    transforms.Resize((512, 512)),
+    transforms.CenterCrop(480),
+    transforms.ToTensor(),
+    transforms.Normalize(
+        mean=[0.485, 0.456, 0.406],  # ImageNet
+        std=[0.229, 0.224, 0.225],
+    ),
+])
+
+idx_to_label = {
+    0: "upright",
+    1: "rotate_90_clockwise",
+    2: "rotate_180",
+    3: "rotate_90_counterclockwise",
+}
+
+# A high confidence threshold (0.99) is applied: predictions below it
+# default to "upright" to minimize false corrections.
+CONFIDENCE_THRESHOLD = 0.99
+
+def predict_orientation(path):
+    img = Image.open(path).convert("RGB")
+    x = preprocess(img).unsqueeze(0).to("cuda")
+    with torch.no_grad():
+        logits = model(x)
+        probs = torch.softmax(logits, dim=1)[0]
+    top1 = int(torch.argmax(probs))
+    conf = float(probs[top1])
+    label = idx_to_label[top1] if conf >= CONFIDENCE_THRESHOLD else "upright"
+    return label, conf, probs.cpu().tolist()
+
+label, conf, all_probs = predict_orientation("crop.jpg")
+print(f"Predicted correction: {label}, confidence: {conf:.3f}")
+```
+
+### Applying Corrections
+
+```python
+from PIL import Image
+
+def apply_correction(img, label):
+    if label == "upright":
+        return img
+    elif label == "rotate_90_clockwise":
+        return img.rotate(-90, expand=True)
+    elif label == "rotate_180":
+        return img.rotate(180, expand=True)
+    elif label == "rotate_90_counterclockwise":
+        return img.rotate(90, expand=True)
+    else:
+        raise ValueError(f"Unknown label: {label}")
+```
+
+## Limitations
+
+- Trained specifically on crops from the [Institutional Books collection](https://huggingface.co/collections/institutional/institutional-books). Performance may degrade on:
+  - Non-book imagery
+  - Heavily stylized or abstract content
+  - Very low-resolution or heavily compressed scans
+- Supports only **multiples of 90°** rotations; does not handle slight skews or arbitrary angle rotations.
+- Expected to work best when:
+  - Crops contain a clear visual object/structure
+  - Background is not overwhelmingly dominant
+- Model assumes images are RGB; grayscale images are internally handled via standard preprocessing but not natively optimized for non-RGB channels.
+
+## Citation
+
+> Citation forthcoming.

weights/weights.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca769dd1135b03e6873ee953b408e460f3cfab909fd45bc78099b4e398670528
+size 213069763