sugiv commited on
Commit
4a73e94
·
verified ·
1 Parent(s): c24b776

Add library requirements, tokenizer loading notes, and UNEXPECTED key explanation

Browse files
Files changed (1) hide show
  1. README.md +24 -1
README.md CHANGED
@@ -42,8 +42,30 @@ Fine-tuned from [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai
42
  ## 🚀 Quick Start
43
 
44
  ```bash
45
- pip install transformers peft torch
46
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
 
48
  ```python
49
  from transformers import AutoModel, AutoTokenizer
@@ -61,6 +83,7 @@ model = PeftModel.from_pretrained(
61
  )
62
  model.eval()
63
 
 
64
  tokenizer = AutoTokenizer.from_pretrained(
65
  "answerdotai/ModernBERT-base",
66
  trust_remote_code=True
 
42
  ## 🚀 Quick Start
43
 
44
  ```bash
45
+ pip install "transformers>=4.48" "peft>=0.14" "huggingface_hub>=0.27" torch
46
  ```
47
+ > **Important:** ModernBERT requires `transformers >= 4.48`. Older versions will fail with `KeyError: 'modernbert'`.
48
+
49
+ ### Requirements
50
+
51
+ | Library | Minimum Version | Notes |
52
+ |---------|----------------|-------|
53
+ | `transformers` | **>= 4.48** | ModernBERT architecture support |
54
+ | `peft` | **>= 0.14** | Compatible `hf_hub_download` API |
55
+ | `huggingface_hub` | **>= 0.27** | No deprecated `use_auth_token` |
56
+ | `torch` | **>= 2.0** | CUDA support |
57
+
58
+ ```bash
59
+ pip install "transformers>=4.48" "peft>=0.14" "huggingface_hub>=0.27" torch
60
+ ```
61
+
62
+ ### Loading Notes
63
+
64
+ **Tokenizer:** Load from the *base model* (`answerdotai/ModernBERT-base`), NOT from this adapter repo. The adapter repo stores LoRA weights only; the tokenizer lives with the base model.
65
+
66
+ **UNEXPECTED keys on load:** When loading `AutoModel.from_pretrained("answerdotai/ModernBERT-base")`, you may see warnings about UNEXPECTED keys (`head.norm.weight`, `head.dense.weight`, `decoder.bias`). These are the base model's **MLM (masked language model) head weights** that exist in the pretrained checkpoint but are not used by `AutoModel` (which loads only the encoder backbone). This is **completely normal** and safe to ignore.
67
+
68
+
69
 
70
  ```python
71
  from transformers import AutoModel, AutoTokenizer
 
83
  )
84
  model.eval()
85
 
86
+ # IMPORTANT: Load tokenizer from BASE MODEL, not from adapter repo
87
  tokenizer = AutoTokenizer.from_pretrained(
88
  "answerdotai/ModernBERT-base",
89
  trust_remote_code=True