---
library_name: transformers
tags: []
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->



## Model Details

### Model Description

- **Developed by:** Mohammad Essam ([metga97](https://huggingface.co/metga97))
- **Model type:** BERT-style encoder
- **Language(s):** Arabic (MSA + Egyptian dialect)
- **License:** MIT
- **Finetuned from model:** [`metga97/Modern-EgyBert-Base`](https://huggingface.co/metga97/Modern-EgyBert-Base)


## Uses

This model is intended to be used for generating sentence embeddings for downstream tasks:

- Sentence similarity
- Semantic retrieval
- Clustering of Arabic sentences
- Intent classification
- Duplicate detection


## How to Get Started with the Model

Use the code below to get started with the model.

```python
from transformers import AutoTokenizer, AutoModel
import torch

tokenizer = AutoTokenizer.from_pretrained("metga97/Modern-EgyBert-Embedding")
model = AutoModel.from_pretrained("metga97/Modern-EgyBert-Embedding")

text = ["الجو النهارده جميل"]
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)

with torch.no_grad():
    outputs = model(**inputs)
    last_hidden = outputs.last_hidden_state

# Mean Pooling
attention_mask = inputs["attention_mask"]
input_mask_expanded = attention_mask.unsqueeze(-1).expand(last_hidden.size()).float()
sum_embeddings = torch.sum(last_hidden * input_mask_expanded, dim=1)
sum_mask = torch.clamp(input_mask_expanded.sum(1), min=1e-9)
sentence_embedding = sum_embeddings / sum_mask

print(sentence_embedding.shape)  # torch.Size([1, 768])
```