Update README.md

README.md

@@ -1,3 +1,142 @@
 ---
 license: apache-2.0
+language:
+- en
+library_name: transformers
+tags:
+- bert
+- scibert
+- scientific-text
+- mirror
+- r-compatible
+base_model: allenai/scibert_scivocab_cased
+pipeline_tag: feature-extraction
 ---
+
+# SciBERT (cased) — safetensors mirror for use from R
+
+This is a format-converted mirror of [`allenai/scibert_scivocab_cased`](https://huggingface.co/allenai/scibert_scivocab_cased), maintained for teaching a course on transformer-based topic modeling in R.
+
+The model itself is unchanged: same architecture, same weights, same tokenizer, same outputs as the upstream original. What's different is the on-disk format and provenance, both of which matter for a teaching context.
+
+## Why this mirror exists
+
+The upstream repo ships `pytorch_model.bin` (PyTorch pickle format) and `vocab.txt`. Both are fine for Python users, but they create friction for R users working through the `torch` (libtorch) and `safetensors` R packages:
+
+- **Pickle weights** require either the `transformers` Python library or the R-torch pickle reader, which has known limitations (cannot remap CUDA-saved tensors to CPU, executes arbitrary code on load, slower to read than safetensors).
+- **`tokenizer.json` is missing**, which forces R code to either depend on a separate WordPiece-tokenization package or to install the Python `tokenizers` library through `reticulate`.
+
+This mirror adds:
+
+- `model.safetensors` — the same weights in [safetensors](https://huggingface.co/docs/safetensors) format, which is device-agnostic, safe (cannot execute code on load), and faster to read than pickle.
+
+Everything else — `config.json`, `vocab.txt`, the model architecture — is identical to the upstream original. The original `pytorch_model.bin` is preserved alongside the safetensors copy so the repo remains a strict superset of the upstream.
+
+## What it is, briefly
+
+SciBERT is BERT-base trained on a corpus of 1.14 million scientific papers from Semantic Scholar (biomedical and computer science). It uses a specialized vocabulary built from scientific text — terms like `protein`, `algorithm`, `mitochondria`, and `gradient` are single tokens rather than fragments, which gives the model a meaningful advantage on scientific text compared to general-purpose BERT.
+
+| Property | Value |
+|----------|-------|
+| Architecture | BERT-base |
+| Parameters | ~110M |
+| Hidden size | 768 |
+| Layers | 12 |
+| Attention heads | 12 |
+| Vocabulary size | 31,116 (cased, scientific) |
+| Max sequence length | 512 tokens |
+| Training data | 1.14M scientific papers (Semantic Scholar) |
+| Case sensitivity | **Cased** (preserves capitalization — important for gene names, chemical formulas, acronyms) |
+
+## Important caveat: this is a base BERT, not a sentence-transformer
+
+SciBERT was trained only on masked language modeling. Its token-level representations are excellent on scientific text, but the **mean-pooled sentence embeddings cluster poorly** — a well-known limitation of base BERT-family models. For sentence-similarity tasks, retrieval, or topic modeling, fine-tuned variants like [`pritamdeka/S-Scibert-snli-multinli-stsb`](https://huggingface.co/pritamdeka/S-Scibert-snli-multinli-stsb) perform substantially better.
+
+SciBERT itself is best used for: token-level tasks (NER, classification fine-tuning), masked language modeling, or as a starting point for further fine-tuning.
+
+## Usage from R
+
+This mirror is set up to work with a pure-R BERT inference pipeline built on top of the `torch` (libtorch) R package, with no Python at runtime:
+
+```r
+source("bert_r.R")
+enc <- load_hf_bert("NetworkIsLife/SciBert_Cased_DAFS")
+
+emb <- embed_texts(enc$model, enc$tokenizer,
+                   c("CRISPR-Cas9 enables targeted gene editing.",
+                     "Glioblastoma exhibits invasive growth patterns."))
+dim(emb)   # 2 x 768
+```
+
+The R loader looks for `model.safetensors` first (this file) and falls back to `pytorch_model.bin` if it isn't found. Since the safetensors file is present, that's the fast path taken.
+
+For long-term reproducibility in course materials, pin to a specific revision:
+
+```r
+enc <- load_hf_bert(
+  "NetworkIsLife/SciBert_Cased_DAFS",
+  weights_path = hfhub::hub_download(
+    "NetworkIsLife/SciBert_Cased_DAFS",
+    "model.safetensors",
+    revision = "MAIN_COMMIT_HASH_HERE"
+  )
+)
+```
+
+Replace `MAIN_COMMIT_HASH_HERE` with the commit hash visible in this repo's commit history.
+
+## Usage from Python (unchanged from upstream)
+
+```python
+from transformers import AutoTokenizer, AutoModel
+tok = AutoTokenizer.from_pretrained("NetworkIsLife/SciBert_Cased_DAFS")
+mod = AutoModel.from_pretrained("NetworkIsLife/SciBert_Cased_DAFS")
+```
+
+## Files in this repo
+
+| File | Source | Purpose |
+|------|--------|---------|
+| `model.safetensors` | converted from upstream `pytorch_model.bin` | model weights, modern format |
+| `pytorch_model.bin` | copied from upstream | model weights, legacy format (kept for compatibility) |
+| `config.json` | copied from upstream | architecture parameters |
+| `vocab.txt` | copied from upstream | WordPiece vocabulary |
+| `README.md` | this file | provenance and usage |
+
+## Provenance and verification
+
+The `model.safetensors` file in this repo was produced by HuggingFace's official `SFconvertbot` (the same automated conversion used across thousands of HuggingFace repos). The conversion is purely a re-serialization — every tensor in the safetensors file is bit-identical to the corresponding tensor in `pytorch_model.bin`. No re-training, no quantization, no precision loss.
+
+You can verify this yourself in Python:
+
+```python
+import torch
+from safetensors.torch import load_file
+
+a = torch.load("pytorch_model.bin", map_location="cpu", weights_only=True)
+b = load_file("model.safetensors")
+assert set(a.keys()) == set(b.keys())
+for k in a:
+    assert torch.equal(a[k], b[k]), f"Mismatch in {k}"
+print("Bit-identical.")
+```
+
+## License and citation
+
+This mirror inherits the upstream license: **Apache 2.0**. If you use this model in academic work, please cite the original SciBERT paper:
+
+```bibtex
+@inproceedings{beltagy-etal-2019-scibert,
+  title = "{SciBERT}: A Pretrained Language Model for Scientific Text",
+  author = "Beltagy, Iz and Lo, Kyle and Cohan, Arman",
+  booktitle = "Proceedings of EMNLP-IJCNLP",
+  year = "2019",
+  url = "https://www.aclweb.org/anthology/D19-1371"
+}
+```
+
+Original model: [`allenai/scibert_scivocab_cased`](https://huggingface.co/allenai/scibert_scivocab_cased) by the Allen Institute for AI.
+
+## Maintenance
+
+This is a teaching artifact for a course on transformer-based topic modeling in R. It will not be updated except to fix conversion errors. For the canonical, maintained version of SciBERT, see the [upstream repo](https://huggingface.co/allenai/scibert_scivocab_cased).