Add comprehensive model card

This PR adds a comprehensive model card for the model.

It includes:
- A link to the paper: [Should We Still Pretrain Encoders with Masked Language Modeling?](https://huggingface.co/papers/2507.00994)
- A link to the project page: [https://hf.co/MLMvsCLM](https://hf.co/MLMvsCLM)
- A link to the GitHub repository: [https://github.com/Nicolas-BZRD/EuroBERT](https://github.com/Nicolas-BZRD/EuroBERT)
- Relevant metadata (`license: apache-2.0`, `library_name: transformers`, `pipeline_tag: feature-extraction`) for proper discoverability on the Hugging Face Hub.
- A basic Python usage example for feature extraction.

This improves the model's documentation and usability.

README.md

@@ -0,0 +1,66 @@
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: feature-extraction
+---
+
+# Should We Still Pretrain Encoders with Masked Language Modeling?
+
+This repository contains a model artifact related to the paper **"[Should We Still Pretrain Encoders with Masked Language Modeling?](https://huggingface.co/papers/2507.00994)"**.
+
+The paper explores the fundamental question of whether Masked Language Modeling (MLM) or Causal Language Modeling (CLM) is more effective for pretraining encoder models to learn high-quality text representations. Through extensive large-scale ablations (30 models from 210M to 1B parameters, 15,000+ fine-tuning runs), the authors find that while MLM generally yields better performance, CLM-trained models are more data-efficient and offer improved fine-tuning stability. A key finding is that a biphasic training strategy (CLM then MLM) achieves optimal performance, especially when initializing from existing pretrained CLM models.
+
+*   **Project Page:** [https://hf.co/MLMvsCLM](https://hf.co/MLMvsCLM)
+*   **Codebase (Optimus Training Library):** The models discussed in the paper were trained using the [Optimus Training Library](https://github.com/Nicolas-BZRD/EuroBERT), a flexible and scalable framework for training language models.
+
+## Model Description
+
+This model is an encoder-only transformer (SLModel architecture) with a hidden size of 1152, 26 hidden layers, and a vocabulary size of 128256, as derived from its `config.json`. It is designed for text representation tasks and feature extraction.
+
+## Usage
+
+This model can be used for feature extraction with the `transformers` library. Since `SLModel` is a custom architecture, `trust_remote_code=True` is required. The model's `torch_dtype` is `bfloat16` as specified in its configuration.
+
+```python
+import torch
+from transformers import AutoModel, AutoTokenizer
+
+# Replace "MLMvsCLM/your-model-id" with the actual repository ID for this model.
+# Example from the research project: "MLMvsCLM/mlm-clm-210m"
+model_id = "MLMvsCLM/mlm-clm-210m" 
+
+tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+model = AutoModel.from_pretrained(model_id, torch_dtype=torch.bfloat16, trust_remote_code=True)
+
+text = "The quick brown fox jumps over the lazy dog."
+inputs = tokenizer(text, return_tensors="pt")
+
+# Move model to GPU if available
+if torch.cuda.is_available():
+    model.to("cuda")
+    inputs = {k: v.to("cuda") for k, v in inputs.items()}
+
+with torch.no_grad():
+    outputs = model(**inputs)
+
+# The last_hidden_state contains the contextualized embeddings for each token.
+# For sentence embeddings, common practice is to average token embeddings or use the CLS token if present.
+last_hidden_state = outputs.last_hidden_state
+print(f"Shape of last_hidden_state: {last_hidden_state.shape}")
+```
+
+## Citation
+
+If you use this model or the findings from the paper in your research, please cite:
+
+```bibtex
+@misc{boizard2025should,
+      title={Should We Still Pretrain Encoders with Masked Language Modeling?}, 
+      author={Nicolas Boizard and Hippolyte Gisserot-Boukhlef and André Martins and Ricardo Rei and Patrick Fernandes and Ayoub Hammal and João Alves and Duarte M. Alves and Céline Hudelot and Etienne Malaboeuf and Emmanuel Malherbe and Fanny Jourdan and Gabriel Hautreux and Kevin El-Haddad and Manuel Faysse and Maxime Peyrard and Nuno M. Guerreiro and Pierre Colombo and Caio Corro},
+      year={2025},
+      eprint={2507.00994},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2507.00994}, 
+}
+```