Mirror of prajjwal1/bert-tiny with attribution and usage docs

.gitattributes

@@ -1,35 +1,8 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
 *.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
 *.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
 *.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+*.tar.gz filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,80 @@
+---
+license: mit
+library_name: transformers
+pipeline_tag: fill-mask
+base_model: prajjwal1/bert-tiny
+language:
+- en
+tags:
+- bert
+- tiny
+- lightweight
+- edge
+- cpu
+- text-embedding
+---
+
+# bert-tiny (mirror)
+
+A 2-layer, 128-hidden BERT — about **4.4M parameters / 17 MB**. Small enough to fine-tune on a
+laptop CPU in minutes, which makes it the go-to model for smoke tests, CI pipelines, unit tests
+for training code, and edge deployment.
+
+> [!NOTE]
+> **This is a mirror.** The weights and tokenizer files here are an unmodified copy of
+> [`prajjwal1/bert-tiny`](https://huggingface.co/prajjwal1/bert-tiny), re-hosted on this profile for reproducibility and
+> convenience. All credit for the original work belongs to its authors. The upstream license
+> (`mit`) is preserved and applies to this copy. If you need the canonical version, please
+> use the upstream repository.
+
+## Specs
+
+| | |
+|---|---|
+| Layers | 2 |
+| Hidden size | 128 |
+| Attention heads | 2 |
+| Parameters | ~4.4M |
+| Vocab | 30,522 (uncased WordPiece) |
+| Disk | ~17 MB |
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModel
+
+tok = AutoTokenizer.from_pretrained("priyaganesh2050/bert-tiny")
+model = AutoModel.from_pretrained("priyaganesh2050/bert-tiny")
+
+out = model(**tok("A tiny BERT for fast experiments.", return_tensors="pt"))
+print(out.last_hidden_state.shape)   # torch.Size([1, 9, 128])
+```
+
+Fine-tuning for classification:
+
+```python
+from transformers import AutoModelForSequenceClassification
+
+model = AutoModelForSequenceClassification.from_pretrained("priyaganesh2050/bert-tiny", num_labels=2)
+```
+
+## When to use this
+
+- **Good for:** CI/CD tests of training loops, hyperparameter search, teaching, edge/mobile,
+  latency-critical baselines.
+- **Not good for:** accuracy-sensitive production NLP. A 2-layer model gives up a lot of quality
+  versus `bert-base`. Use it as a baseline, then scale up.
+
+## Citation
+
+The tiny BERT variants come from the well-read-students line of work:
+
+```bibtex
+@misc{turc2019,
+  title  = {Well-Read Students Learn Better: On the Importance of Pre-training Compact Models},
+  author = {Turc, Iulia and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
+  year   = {2019},
+  eprint = {1908.08962},
+  archivePrefix = {arXiv}
+}
+```

config.json

@@ -0,0 +1 @@
+{"hidden_size": 128, "hidden_act": "gelu", "initializer_range": 0.02, "vocab_size": 30522, "hidden_dropout_prob": 0.1, "num_attention_heads": 2, "type_vocab_size": 2, "max_position_embeddings": 512, "num_hidden_layers": 2, "intermediate_size": 512, "attention_probs_dropout_prob": 0.1}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dab2c2bddcfb48ea430ef63fd76d46d67d704487844d967256a50dd7d7fd0a66
+size 17756393