| | --- |
| | library_name: transformers |
| | tags: [] |
| | --- |
| | |
| | # Model Card for ModernBERT-Wikt-base-verb |
| |
|
| |
|
| | ### Model Description |
| |
|
| | This model is an English language model based on ModernBERT-base, fine-tuned using verb examples from English Wiktionary via supervised contrastive learning. |
| | The fine-tuning improves token-level semantic representations, particularly for tasks like Word-in-Context (WiC) and Word Sense Disambiguation (WSD). |
| |
|
| | Although trained on verbs, the model shows enhanced representation quality across the lexicon. |
| |
|
| |
|
| | - **Developed by:** Anna Mosolova, Marie Candito, Carlos Ramisch |
| | - **Funded by:** [ANR Selexini](https://selexini.lis-lab.fr) |
| | - **Model type:** BERT-based transformer (ModernBERT) |
| | - **Language:** English |
| | - **License:** MIT |
| | - **Finetuned from model:** [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) |
| |
|
| | ### Model Sources |
| |
|
| |
|
| | - **Repository:** [https://github.com/anya-bel/contrastive_learning_transfer](https://github.com/anya-bel/contrastive_learning_transfer) |
| | - **Paper:** [Raffinage des représentations des tokens dans les modèles de langue pré-entraînés avec l’apprentissage contrastif : une étude entre modèles et entre langues](https://coria-taln-2025.lis-lab.fr/wp-content/uploads/2025/06/CORIA-TALN_2025_paper_139.pdf) |
| |
|
| | ## Uses |
| |
|
| | The model is intended for extracting token-level embeddings for English, with improved sense separation. |
| |
|
| |
|
| | ## How to Get Started with the Model |
| |
|
| | ``` |
| | from transformers import AutoTokenizer, AutoModel |
| | |
| | tokenizer = AutoTokenizer.from_pretrained("answerdotai/ModernBERT-base") |
| | model = AutoModel.from_pretrained("annamos/ModernBERT-Wikt-base-verb") |
| | sentence = 'You should knock before you enter' |
| | tokenized = tokenizer(sentence, return_tensors='pt') |
| | embeddings = model(**tokenized)[0] |
| | ``` |
| |
|
