---
license: apache-2.0
language:
- it
base_model:
- dbmdz/bert-base-italian-uncased
tags:
- legal
- italian
- infocube
metrics:
- perplexity
pipeline_tag: fill-mask
library_name: transformers
---
# Model Card for Model ID

This model is a BERT-based Masked Language Model fine-tuned on Italian legal texts. It is designed to predict masked tokens in legal documents and capture domain-specific semantic and syntactic structures.



### Model Description

This model is fine-tuned from `dbmdz/bert-base-italian-uncased` using **Masked Language Modeling (MLM) with Whole Word Masking (WWM)**.  
WWM ensures that all subword tokens of a selected word are masked together, encouraging the model to learn deeper contextual representations, especially for complex legal terminology.



- **Developed by:** [Mohammad Mahdi Heydari Asl](https://huggingface.co/HYDARIM7) / infocube]  
- **Model type:** Transformer, BERT-based Masked Language Model  
- **Language(s):** Italian  
- **License:** Apache-2.0  
- **Finetuned from model:** `dbmdz/bert-base-italian-uncased`  


## Uses


### Direct Use

The model can be used for:
- Predicting masked tokens in Italian legal texts (`[MASK]` prediction)  
- Embedding legal text for downstream NLP tasks  
- Transfer learning for other Italian legal NLP applications


## Bias, Risks, and Limitations

- Not suitable for general-purpose Italian NLP outside legal text.  


### Recommendations

Users should verify outputs and avoid relying on predictions for legal decision-making without expert supervision.


## How to Get Started with the Model

```python
from transformers import AutoTokenizer, AutoModelForMaskedLM
import torch

model_name = "InfocubeSrl/LexCube"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForMaskedLM.from_pretrained(model_name)

text = "La legge [MASK] approvata dal parlamento."
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)

mask_index = (inputs["input_ids"][0] == tokenizer.mask_token_id).nonzero()[0]
predicted_id = outputs.logits[0, mask_index].argmax()
predicted_token = tokenizer.decode(predicted_id)

print("Prediction:", predicted_token)
```


### Training Data

- **Source:** Provided by *Infocube*, 
- **Size:** 15,646 documents  
- **Language:** Italian  
- **Domain:** Legal and administrative texts
  - Formal and technical legal language  
  - Frequent references to laws, decrees, and legislative articles  
  - Structured format with numbered provisions and cross-citations  
  - Avg. length: ~909 words (≈2,193 tokens per document); some documents exceed 11k tokens  
- **Confidentiality:** Raw dataset cannot be shared due to contractual agreements, but it has been statistically and linguistically analyzed for research