Upload 10 files

README.md

@@ -1,3 +1,197 @@
 ---
+language:
+- he
 license: apache-2.0
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- biblical-hebrew
+- digital-humanities
+- inner-biblical-parallels
+base_model: imvladikon/sentence-transformers-alephbert
+datasets:
+- davidmsmiley/tomim
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+model-index:
+- name: MiqraBERT
+  results:
+  - task:
+      type: sentence-similarity
+      name: Semantic Similarity
+    dataset:
+      name: "T'OMIM"
+      type: davidmsmiley/tomim
+    metrics:
+    - type: f1
+      value: 0.980
+      name: F1 (threshold=0.53)
+    - type: recall_at_10
+      value: 0.728
+      name: Recall@10 (all pairs)
+    - type: recall_at_10
+      value: 0.871
+      name: Recall@10 (narrative)
+widget:
+- source_sentence: "וַיַּעַשׂ הַיָּשָׁר בְּעֵינֵי יְהוָה כְּכֹל אֲשֶׁר־עָשָׂה עֻזִּיָּהוּ אָבִיו רַק לֹא־בָא אֶל־הֵיכַל יְהוָה וְעוֺד הָעָם מַשְׁחִיתִים"
+  sentences:
+  - "וַיַּעַשׂ הַיָּשָׁר בְּעֵינֵי יְהוָה כְּכֹל אֲשֶׁר־עָשָׂה עֻזִיָּהוּ אָבִיו עָשָׂה"
+  - "וְהִנֵּה שֶׁבַע שִׁבֳּלִים צְנֻמוֺת דַּקּוֺת שְׁדֻפוֺת קָדִים צֹמְחוֺת אַחֲרֵיהֶם"
+  - "יִשָּׁעֵן עַל־בֵּיתוֺ וְלֹא יַעֲמֹד יַחֲזִיק בּוֺ וְלֹא יָקוּם"
 ---
+
+# MiqraBERT
+
+A [sentence-transformers](https://www.sbert.net) model finetuned from [AlephBERT](https://huggingface.co/imvladikon/sentence-transformers-alephbert) for detecting parallel passages in the Hebrew Bible. It maps Biblical Hebrew verses to 768-dimensional embeddings where cosine similarity reflects textual parallelism — high scores indicate genuine synoptic parallels, low scores indicate unrelated text.
+
+*MiqraBERT* derives from Hebrew מִקְרָא (*miqra*, "scripture").
+
+## Model Details
+
+- **Developed by:** David M. Smiley, University of Notre Dame
+- **Model type:** Sentence Transformer (BERT encoder + mean pooling)
+- **Language:** Biblical Hebrew (vocalized, with niqqud)
+- **Base model:** [imvladikon/sentence-transformers-alephbert](https://huggingface.co/imvladikon/sentence-transformers-alephbert) (AlephBERT)
+- **Finetuned on:** [T'OMIM](https://huggingface.co/datasets/davidmsmiley/tomim) — 1,650 Biblical Hebrew verse pairs ([Zenodo](https://doi.org/10.5281/zenodo.19135731))
+- **Output:** 768 dimensions, cosine similarity
+- **Max sequence length:** 512 tokens
+- **License:** Apache 2.0
+- **Paper:** [arXiv:2506.24117](https://arxiv.org/abs/2506.24117)
+
+## Usage
+
+### Sentence Transformers
+
+```bash
+pip install -U sentence-transformers
+```
+
+```python
+from sentence_transformers import SentenceTransformer
+
+model = SentenceTransformer("davidmsmiley/miqrabert")
+
+# 2 Kgs 18:13 and its synoptic parallel Isa 36:1
+parallel_a = "וּבְאַרְבַּע עֶשְׂרֵה שָׁנָה לַמֶּלֶךְ חִזְקִיָּהוּ עָלָה סַנְחֵרִיב מֶלֶךְ־אַשּׁוּר עַל כָּל־עָרֵי יְהוּדָה הַבְּצֻרוֺת וַיִּתְפְּשֵׂם"
+parallel_b = "וַיְהִי בְּאַרְבַּע עֶשְׂרֵה שָׁנָה לַמֶּלֶךְ חִזְקִיָּהוּ עָלָה סַנְחֵרִיב מֶלֶךְ־אַשּׁוּר עַל־כָּל־עָרֵי יְהוּדָה הַבְּצֻרוֺת וַיִּתְפְּשֵׂם"
+unrelated  = "וְהִנֵּה שֶׁבַע שִׁבֳּלִים צְנֻמוֺת דַּקּוֺת שְׁדֻפוֺת קָדִים צֹמְחוֺת אַחֲרֵיהֶם"
+
+embeddings = model.encode([parallel_a, parallel_b, unrelated])
+similarities = model.similarity(embeddings, embeddings)
+# parallel_a ↔ parallel_b: ~0.98 (near-verbatim parallel)
+# parallel_a ↔ unrelated:  ~0.05 (no relationship)
+```
+
+### Using Transformers Directly
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModel
+
+tokenizer = AutoTokenizer.from_pretrained("davidmsmiley/miqrabert")
+model = AutoModel.from_pretrained("davidmsmiley/miqrabert")
+
+def encode(texts):
+    inputs = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
+    with torch.no_grad():
+        output = model(**inputs)
+    mask = inputs["attention_mask"].unsqueeze(-1)
+    embeddings = (output.last_hidden_state * mask).sum(1) / mask.sum(1)
+    return torch.nn.functional.normalize(embeddings, p=2, dim=1)
+
+emb = encode(["וַיַּעַשׂ הַיָּשָׁר בְּעֵינֵי יְהוָה", "וַיַּעַשׂ הָרַע בְּעֵינֵי יְהוָה"])
+similarity = torch.nn.functional.cosine_similarity(emb[0], emb[1], dim=0)
+```
+
+## Intended Uses
+
+**Use for:** measuring semantic similarity between Biblical Hebrew verse pairs; identifying candidate parallel passages across the Hebrew Bible; supporting computational research on inner-biblical allusion and textual reuse.
+
+**Not designed for:** Modern Hebrew, Rabbinic Hebrew, or Aramaic text. Not optimized for poetic parallelism (see Limitations). Outputs continuous similarity scores — not a binary classifier.
+
+## Training
+
+### Data
+
+[T'OMIM](https://huggingface.co/datasets/davidmsmiley/tomim) contains 825 parallel and 825 non-parallel Biblical Hebrew verse pairs. Parallels include 556 narrative pairs from Chronicles // Samuel-Kings and 269 poetic pairs from published parallelism studies (Berlin, Fokkelman, Kugel, Tsumura). Negatives are random pairs sampled from the full Hebrew Bible.
+
+### Procedure
+
+Cosine similarity regression via [CosineSimilarityLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) (MSE). Both verses pass through a shared encoder, are mean-pooled to 768-dim embeddings, and compared via cosine similarity against target labels (1.0 = parallel, 0.0 = non-parallel). This checkpoint uses a 70/15/15 train-validation-test split (1,155 / 247 / 248 pairs), selected from seven configurations (50%–90%) as the optimal balance of separation quality and test set size. Stability validated across 10 random seeds (70 models total).
+
+### Hyperparameters
+
+- **Epochs:** 2
+- **Batch size:** 16
+- **Learning rate:** 5e-05 (linear schedule)
+- **Optimizer:** AdamW
+- **Seed:** 42
+- **Hardware:** NVIDIA T4 GPU (~36 seconds)
+
+### Framework Versions
+
+- Sentence Transformers 5.2.0 / Transformers 4.57.3 / PyTorch 2.9.0+cu126
+
+## Evaluation
+
+### Test Set Performance
+
+| Metric | Score |
+|:-------|:------|
+| Wasserstein Distance | 0.772 [0.735, 0.809] |
+| Overlap Coefficient | 0.046 |
+| F1 (threshold = 0.53) | 0.980 |
+| Precision / Recall | 0.984 / 0.976 |
+| Mean cosine sim (parallel) | 0.880 |
+| Mean cosine sim (non-parallel) | 0.108 |
+
+Wasserstein Distance (WD) measures distributional separation between parallel and non-parallel similarity scores; higher is better. Overlap Coefficient (OVL) measures the proportion of ambiguous space where distributions intersect; lower is better. The unfinetuned AlephBERT baseline achieves WD = 0.276 and OVL = 0.240.
+
+### Retrieval (Recall@k)
+
+Each query verse is searched against all 68,125 verse and half-verse vectors in the Hebrew Bible ([BHSA](https://etcbc.github.io/bhsa/) corpus). Recall@k measures how often the true parallel appears in the top-k results.
+
+| Model | Recall@10 (all) | Recall@10 (narrative) | Recall@10 (poetic) |
+|:------|:---------------:|:---------------------:|:------------------:|
+| **MiqraBERT-70p** | **0.728** | **0.871** | 0.089 |
+| BEREL-70p | 0.704 | 0.831 | 0.137 |
+| DictaLM-70p | 0.751 | 0.914 | 0.024 |
+
+MiqraBERT is selected as the primary model for its balance across metrics: strong narrative recall, stable training, and the smallest parameter footprint (~110M vs. 7.25B for DictaLM). Full model comparison in the [paper](https://arxiv.org/abs/2506.24117).
+
+## Limitations
+
+- **Narrative focus:** Trained primarily on Chronicles // Samuel-Kings synoptic parallels. Recall@10 for poetic parallelism is only 8.9% — a structural limitation of mean-pooled embeddings for texts with little lexical overlap.
+- **Biblical Hebrew only:** Not evaluated on Modern Hebrew, Rabbinic Hebrew, unvocalized text, or other Semitic languages.
+- **Training scope:** May underperform on intertextual relationships not represented in training (allusions, type-scenes, formulaic speech).
+
+## Citation
+
+```bibtex
+@article{smiley2025intertextual,
+    title  = {Intertextual Parallel Detection in Biblical Hebrew: A Transformer-Based Benchmark},
+    author = {Smiley, David M.},
+    journal = {arXiv preprint arXiv:2506.24117},
+    year   = {2025},
+    url    = {https://arxiv.org/abs/2506.24117}
+}
+```
+
+### Upstream Models
+
+```bibtex
+@inproceedings{reimers2019sentencebert,
+    title     = {Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks},
+    author    = {Reimers, Nils and Gurevych, Iryna},
+    booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing},
+    year      = {2019}
+}
+
+@article{seker2021alephbert,
+    title  = {AlephBERT: A Hebrew Large Pre-Trained Language Model to Start-off Your Hebrew NLP Application With},
+    author = {Seker, Amit and Bandel, Elron and Bareket, Dan and Brusilovsky, Idan and Greenfeld, Refael Shaked and Tsarfaty, Reut},
+    journal = {arXiv preprint arXiv:2104.04052},
+    year   = {2021}
+}
+```

config.json

@@ -0,0 +1,25 @@
+{
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.57.3",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 52000
+}

config_sentence_transformers.json

@@ -0,0 +1,14 @@
+{
+  "__version__": {
+    "sentence_transformers": "5.2.0",
+    "transformers": "4.57.3",
+    "pytorch": "2.9.0+cu126"
+  },
+  "model_type": "SentenceTransformer",
+  "prompts": {
+    "query": "",
+    "document": ""
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84794ec60752ba964055437944a8345d51a29a0d9e73699b62d87cc2d305e3bb
+size 503928680

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+    "max_seq_length": 512,
+    "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,66 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_len": 512,
+  "max_length": 512,
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}