Add base_model_prefix="model" on PreTrainedModel

This allows you to load the AutoModel, even though the model checkpoint is for the Masked Language modeling head. In short: the base_model_prefix is what it searches for in the subclasses like NeoBERTLMHead and NeoBERTForSequenceClassification

model.py

@@ -199,6 +199,7 @@ class EncoderBlock(nn.Module):
 
 class NeoBERTPreTrainedModel(PreTrainedModel):
     config_class = NeoBERTConfig
+    base_model_prefix = "model"
     _supports_cache_class = True
 
     def _init_weights(self, module):