Upload folder using huggingface_hub

README.md

@@ -0,0 +1,19 @@
+# Prosody Page Classifiers
+
+Three finetuned models for binary page classification in the
+[Princeton Prosody Archive](https://prosody.princeton.edu/) corpus. Labels are
+`TU` (0) and `non-TU` (1). Each subfolder is independently loadable
+— use whichever modality you have inputs for.
+
+| Folder | Model |
+|--------|-------|
+| `distilbert-text/` | DistilBERT text classifier (text-only) |
+| `vit-image/` | ViT-base image classifier (image-only) |
+| `gated-fusion/` | Gated-fusion multimodal classifier (text + image) |
+
+- **`distilbert-text/`** and **`vit-image/`** are standard Hugging Face repos
+  (`AutoModelForSequenceClassification` / `AutoModelForImageClassification`).
+- **`gated-fusion/`** is a custom multimodal model; load it via the bundled
+  `modeling_gatedfusion.py` (see that folder's README).
+
+See each subfolder's `README.md` for a copy-paste usage snippet.

distilbert-text/README.md

@@ -0,0 +1,24 @@
+# Prosody DistilBERT Text Classifier
+
+Fine-tuned `distilbert-base-uncased` for binary page classification in the
+[Princeton Prosody Archive](https://prosody.princeton.edu/) corpus, using page
+**text** (OCR transcription) only.
+
+- **Classes:** `TU` (0), `non-TU` (1)
+- **Architecture:** `DistilBertForSequenceClassification` (HF-native)
+
+```python
+from transformers import AutoModelForSequenceClassification, AutoTokenizer
+import torch
+
+model = AutoModelForSequenceClassification.from_pretrained("./distilbert-text")
+tok   = AutoTokenizer.from_pretrained("./distilbert-text")
+
+enc = tok("a line of verse ...", truncation=True, max_length=512, return_tensors="pt")
+with torch.no_grad():
+    probs = model(**enc).logits.softmax(-1)[0]
+print({model.config.id2label[i]: float(p) for i, p in enumerate(probs)})
+```
+
+`label_encoder.pkl` is the original sklearn `LabelEncoder` (`class1`->0, `class2`->1)
+kept for provenance.

distilbert-text/config.json

@@ -0,0 +1,35 @@
+{
+  "activation": "gelu",
+  "architectures": [
+    "DistilBertForSequenceClassification"
+  ],
+  "attention_dropout": 0.1,
+  "bos_token_id": null,
+  "dim": 768,
+  "dropout": 0.1,
+  "dtype": "float32",
+  "eos_token_id": null,
+  "hidden_dim": 3072,
+  "id2label": {
+    "0": "TU",
+    "1": "non-TU"
+  },
+  "initializer_range": 0.02,
+  "label2id": {
+    "TU": 0,
+    "non-TU": 1
+  },
+  "max_position_embeddings": 512,
+  "model_type": "distilbert",
+  "n_heads": 12,
+  "n_layers": 6,
+  "pad_token_id": 0,
+  "problem_type": "single_label_classification",
+  "qa_dropout": 0.1,
+  "seq_classif_dropout": 0.2,
+  "sinusoidal_pos_embds": false,
+  "tie_weights_": true,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.11.0",
+  "vocab_size": 30522
+}

distilbert-text/label_encoder.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b117610370253f52309d0663253878cd20c9674e2d87456e467e3ca7ba5ecf18
+size 283

distilbert-text/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a299309f21dc059ae68073b24c121e006abb3c69e2aa1a39537be15c27061b02
+size 267832560

distilbert-text/tokenizer_config.json

@@ -0,0 +1,18 @@
+{
+  "backend": "tokenizers",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "is_local": true,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "DistilBertTokenizer",
+  "unk_token": "[UNK]"
+}

gated-fusion/README.md

@@ -0,0 +1,29 @@
+# Prosody Gated-Fusion Multimodal Classifier
+
+Multimodal (text + page-image) binary classifier for the
+[Princeton Prosody Archive](https://prosody.princeton.edu/) corpus. Fuses a
+**DistilBERT** text encoder and a **ViT-base** image encoder with a learnable
+**gate** that weights the two modalities per example.
+
+- **Classes:** `TU` (0), `non-TU` (1)
+- **Text encoder:** `distilbert-base-uncased` · **Vision encoder:** `google/vit-base-patch16-224`
+- **Fusion:** per-modality projection to 512-d, a sigmoid gate over the
+  concatenated features, a residual combination, and a linear head.
+
+```python
+import sys; sys.path.insert(0, ".")   # so modeling_gatedfusion.py is importable
+from modeling_gatedfusion import GatedFusionClassifier, GatedFusionProcessor
+import torch
+
+model = GatedFusionClassifier.from_pretrained(".")
+proc  = GatedFusionProcessor.from_pretrained(".")
+
+batch = proc(text="a line of verse ...", image="page.png")
+with torch.no_grad():
+    probs = model(**batch).softmax(-1)[0]
+print({model_labels[i]: float(p) for i, p in enumerate(probs)})  # see config.json id2label
+```
+
+`modeling_gatedfusion.py` rebuilds the DistilBERT/ViT encoders from their base
+configs and loads every weight from `model.safetensors`, so no separate
+base-weight files are needed.

gated-fusion/config.json

@@ -0,0 +1,21 @@
+{
+  "model_type": "gated_fusion_multimodal",
+  "architectures": [
+    "GatedFusionClassifier"
+  ],
+  "text_model": "distilbert-base-uncased",
+  "vision_model": "google/vit-base-patch16-224",
+  "text_dim": 768,
+  "vision_dim": 768,
+  "fusion_dim": 512,
+  "num_classes": 2,
+  "max_length": 512,
+  "id2label": {
+    "0": "TU",
+    "1": "non-TU"
+  },
+  "label2id": {
+    "TU": 0,
+    "non-TU": 1
+  }
+}

gated-fusion/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf3873da40b9e678fa3da6dba6fe19f143011b187bdb4003cc1021cdc0024b3a
+size 616302144

gated-fusion/modeling_gatedfusion.py

@@ -0,0 +1,136 @@
+"""
+Standalone definition of the Prosody gated-fusion multimodal classifier
+(text = DistilBERT, vision = ViT-base) plus a `from_pretrained`-style loader.
+
+This file is meant to travel *inside* the Hugging Face repo alongside
+`config.json` and `model.safetensors`. It is fully self-contained:
+
+  * The DistilBERT and ViT encoders are rebuilt from their standard base
+    configurations (distilbert-base-uncased / google/vit-base-patch16-224),
+    which have identical layer shapes to the trained sub-encoders.
+  * All weights — including the encoders — are then loaded from
+    `model.safetensors`, so NO separate base-weight files are required.
+
+Usage:
+    from modeling_gatedfusion import GatedFusionClassifier, GatedFusionProcessor
+    model     = GatedFusionClassifier.from_pretrained("path/to/repo")
+    processor = GatedFusionProcessor.from_pretrained("path/to/repo")
+
+    batch = processor(text="a line of verse", image="page.png")
+    logits = model(**batch)
+"""
+
+import json
+import os
+
+import torch
+import torch.nn as nn
+from PIL import Image
+from safetensors.torch import load_file
+from transformers import (
+    DistilBertConfig, DistilBertModel, DistilBertTokenizer,
+    ViTConfig, ViTModel, ViTImageProcessor,
+)
+
+
+class GatedFusionClassifier(nn.Module):
+    """Learnable gating mechanism to balance text and vision modalities."""
+
+    def __init__(self, distilbert_model, vit_model, num_classes=2, fusion_dim=512):
+        super().__init__()
+        self.distilbert = distilbert_model
+        self.vit = vit_model
+
+        self.text_dim = self.distilbert.config.hidden_size
+        self.vision_dim = self.vit.config.hidden_size
+
+        # Project each modality to a common dimension
+        self.text_projection = nn.Linear(self.text_dim, fusion_dim)
+        self.vision_projection = nn.Linear(self.vision_dim, fusion_dim)
+
+        # Gating mechanism
+        self.gate = nn.Sequential(
+            nn.Linear(fusion_dim * 2, fusion_dim),
+            nn.ReLU(),
+            nn.Linear(fusion_dim, 2),
+            nn.Sigmoid(),
+        )
+
+        self.classifier = nn.Linear(fusion_dim, num_classes)
+        self.dropout = nn.Dropout(0.1)
+
+    def forward(self, input_ids, attention_mask, pixel_values):
+        text_features = self.distilbert(
+            input_ids=input_ids, attention_mask=attention_mask
+        ).last_hidden_state[:, 0]
+        vision_features = self.vit(pixel_values=pixel_values).last_hidden_state[:, 0]
+
+        text_proj = self.text_projection(text_features)
+        vision_proj = self.vision_projection(vision_features)
+
+        gates = self.gate(torch.cat([text_proj, vision_proj], dim=1))
+        gated_text = text_proj * gates[:, 0:1]
+        gated_vision = vision_proj * gates[:, 1:2]
+
+        # Combine gated modalities with a residual connection
+        fused = gated_text + gated_vision + text_proj + vision_proj
+        fused = self.dropout(fused)
+        return self.classifier(fused)
+
+    @classmethod
+    def from_pretrained(cls, repo_dir, map_location="cpu"):
+        """Rebuild the architecture from config.json and load model.safetensors."""
+        with open(os.path.join(repo_dir, "config.json")) as f:
+            cfg = json.load(f)
+
+        # Rebuild encoders from base configs (architecture only — weights come
+        # from the safetensors file below). Defaults match the base checkpoints.
+        distilbert = DistilBertModel(DistilBertConfig())
+        vit = ViTModel(ViTConfig())
+
+        model = cls(
+            distilbert,
+            vit,
+            num_classes=cfg.get("num_classes", 2),
+            fusion_dim=cfg.get("fusion_dim", 512),
+        )
+
+        state = load_file(os.path.join(repo_dir, "model.safetensors"), device=map_location)
+        model.load_state_dict(state, strict=True)  # raises if any key mismatches
+        model.eval()
+        return model
+
+
+class GatedFusionProcessor:
+    """Bundles the DistilBERT tokenizer and ViT image processor."""
+
+    def __init__(self, tokenizer, image_processor, max_length=512):
+        self.tokenizer = tokenizer
+        self.image_processor = image_processor
+        self.max_length = max_length
+
+    @classmethod
+    def from_pretrained(cls, repo_dir):
+        with open(os.path.join(repo_dir, "config.json")) as f:
+            cfg = json.load(f)
+        tokenizer = DistilBertTokenizer.from_pretrained(repo_dir)
+        image_processor = ViTImageProcessor.from_pretrained(repo_dir)
+        return cls(tokenizer, image_processor, max_length=cfg.get("max_length", 512))
+
+    def __call__(self, text, image):
+        """`image` may be a path or a PIL.Image."""
+        if isinstance(image, str):
+            image = Image.open(image).convert("RGB")
+        enc = self.tokenizer(
+            text,
+            truncation=True,
+            padding="max_length",
+            max_length=self.max_length,
+            return_tensors="pt",
+        )
+        pixel_values = self.image_processor(image, return_tensors="pt")["pixel_values"]
+        return {
+            "input_ids": enc["input_ids"],
+            "attention_mask": enc["attention_mask"],
+            "pixel_values": pixel_values,
+        }

gated-fusion/preprocessor_config.json

@@ -0,0 +1,22 @@
+{
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "ViTImageProcessor",
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 224,
+    "width": 224
+  }
+}

gated-fusion/tokenizer_config.json

@@ -0,0 +1,15 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "DistilBertTokenizer",
+  "unk_token": "[UNK]"
+}

vit-image/README.md

@@ -0,0 +1,23 @@
+# Prosody ViT Image Classifier
+
+Fine-tuned `google/vit-base-patch16-224` for binary page classification in the
+[Princeton Prosody Archive](https://prosody.princeton.edu/) corpus, using the
+scanned page **image** only.
+
+- **Classes:** `TU` (0), `non-TU` (1)
+- **Architecture:** `ViTForImageClassification` (HF-native)
+
+```python
+from transformers import AutoModelForImageClassification, AutoImageProcessor
+from PIL import Image
+import torch
+
+model = AutoModelForImageClassification.from_pretrained("./vit-image")
+proc  = AutoImageProcessor.from_pretrained("./vit-image")
+
+img = Image.open("page.png").convert("RGB")
+inp = proc(img, return_tensors="pt")
+with torch.no_grad():
+    probs = model(**inp).logits.softmax(-1)[0]
+print({model.config.id2label[i]: float(p) for i, p in enumerate(probs)})
+```

vit-image/config.json

@@ -0,0 +1,32 @@
+{
+  "architectures": [
+    "ViTForImageClassification"
+  ],
+  "attention_probs_dropout_prob": 0.0,
+  "dtype": "float32",
+  "encoder_stride": 16,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.0,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "TU",
+    "1": "non-TU"
+  },
+  "image_size": 224,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "TU": 0,
+    "non-TU": 1
+  },
+  "layer_norm_eps": 1e-12,
+  "model_type": "vit",
+  "num_attention_heads": 12,
+  "num_channels": 3,
+  "num_hidden_layers": 12,
+  "patch_size": 16,
+  "pooler_act": "tanh",
+  "pooler_output_size": 768,
+  "qkv_bias": true,
+  "transformers_version": "5.11.0"
+}

vit-image/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08d5b964ecef0e4178107b9896d5e5cb83c629829465cc9a0ecb69600b4909f8
+size 343223968

vit-image/preprocessor_config.json

@@ -0,0 +1,22 @@
+{
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "ViTImageProcessor",
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 224,
+    "width": 224
+  }
+}