Add library requirements, tokenizer loading notes, and UNEXPECTED key explanation
Browse files
README.md
CHANGED
|
@@ -42,8 +42,30 @@ Fine-tuned from [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai
|
|
| 42 |
## 🚀 Quick Start
|
| 43 |
|
| 44 |
```bash
|
| 45 |
-
pip install transformers peft torch
|
| 46 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
```python
|
| 49 |
from transformers import AutoModel, AutoTokenizer
|
|
@@ -61,6 +83,7 @@ model = PeftModel.from_pretrained(
|
|
| 61 |
)
|
| 62 |
model.eval()
|
| 63 |
|
|
|
|
| 64 |
tokenizer = AutoTokenizer.from_pretrained(
|
| 65 |
"answerdotai/ModernBERT-base",
|
| 66 |
trust_remote_code=True
|
|
|
|
| 42 |
## 🚀 Quick Start
|
| 43 |
|
| 44 |
```bash
|
| 45 |
+
pip install "transformers>=4.48" "peft>=0.14" "huggingface_hub>=0.27" torch
|
| 46 |
```
|
| 47 |
+
> **Important:** ModernBERT requires `transformers >= 4.48`. Older versions will fail with `KeyError: 'modernbert'`.
|
| 48 |
+
|
| 49 |
+
### Requirements
|
| 50 |
+
|
| 51 |
+
| Library | Minimum Version | Notes |
|
| 52 |
+
|---------|----------------|-------|
|
| 53 |
+
| `transformers` | **>= 4.48** | ModernBERT architecture support |
|
| 54 |
+
| `peft` | **>= 0.14** | Compatible `hf_hub_download` API |
|
| 55 |
+
| `huggingface_hub` | **>= 0.27** | No deprecated `use_auth_token` |
|
| 56 |
+
| `torch` | **>= 2.0** | CUDA support |
|
| 57 |
+
|
| 58 |
+
```bash
|
| 59 |
+
pip install "transformers>=4.48" "peft>=0.14" "huggingface_hub>=0.27" torch
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
### Loading Notes
|
| 63 |
+
|
| 64 |
+
**Tokenizer:** Load from the *base model* (`answerdotai/ModernBERT-base`), NOT from this adapter repo. The adapter repo stores LoRA weights only; the tokenizer lives with the base model.
|
| 65 |
+
|
| 66 |
+
**UNEXPECTED keys on load:** When loading `AutoModel.from_pretrained("answerdotai/ModernBERT-base")`, you may see warnings about UNEXPECTED keys (`head.norm.weight`, `head.dense.weight`, `decoder.bias`). These are the base model's **MLM (masked language model) head weights** that exist in the pretrained checkpoint but are not used by `AutoModel` (which loads only the encoder backbone). This is **completely normal** and safe to ignore.
|
| 67 |
+
|
| 68 |
+
|
| 69 |
|
| 70 |
```python
|
| 71 |
from transformers import AutoModel, AutoTokenizer
|
|
|
|
| 83 |
)
|
| 84 |
model.eval()
|
| 85 |
|
| 86 |
+
# IMPORTANT: Load tokenizer from BASE MODEL, not from adapter repo
|
| 87 |
tokenizer = AutoTokenizer.from_pretrained(
|
| 88 |
"answerdotai/ModernBERT-base",
|
| 89 |
trust_remote_code=True
|