basilepp19/cruciverb-it-IT5-full

This model card is for Model 3 from the UNIBA system, which participated in the Cruciverb-IT task at EVALITA 2026. Unlike models that use partial information, this model serves as a standard baseline for answering Italian crossword clues based purely on the clue text and target length.

---

Model Card: uniba/cruciverb-it-IT5-partial

Model Details

• Developed by: Pierpaolo Basile, Department of Computer Science, University of Bari Aldo Moro.
• Model Type: Encoder-Decoder Transformer.
• Language(s): Italian.
• Base Model: IT5 Large
• Task: Crossword Clue Answering (Subtask 1 of Cruciverb-IT).
• License: Creative Commons Attribution 4.0 International (CC BY 4.0).

Uses

Direct Use

Model 3 is designed to generate candidate answers for Italian crossword clues when no partial grid information is available. It takes the clue and the total answer length as inputs.

Out-of-Scope Use

• Solving crosswords that require conditioning on existing grid characters (for which Model 2 is better suited).
• Use in non-Italian language contexts.

Training Details

Training Data

• Source: Trained exclusively on the original training dataset provided by the EVALITA 2026 organizers (374,766 pairs) without additional external resources like dictionaries.
• Constraint: This is a "constrained" run model, meaning it avoids the use of large proprietary models or external lexical resources.

Training Procedure

• Input Format: Plain text without special tokens: "Trova la soluzione. Lunghezza soluzione: {0}. Indizio: {1}", where {0} is the answer length and {1} is the clue.

• Hyperparameters:

• Learning Rate: .

• Batch Size: 32.

• Weight Decay: 0.01.

• Epochs: 10 (Notably higher than models using partial answers due to smaller dataset size).

• Hardware: Single NVIDIA RTX A6000 with 48GB VRAM.

Evaluation

Metrics

The model was evaluated on Subtask 1 using accuracy at top 1 (acc@1), accuracy at top 10 (acc@10), and Mean Reciprocal Rank (MRR).

Results (Subtask 1)

Model 3 (identified as RUN1 in the paper results) performed as follows:

Metric	Score
acc@1	0.36
acc@10	0.54
MRR	0.41

Bias, Risks, and Limitations

• Lower Performance: Model 3 performs significantly worse than Model 2 (which exploits partial solutions) in realistic solving scenarios.
• Generalization: Its ability to generalize is limited by the use of only task-provided training data.
• Baseline Status: It serves as a baseline for the UNIBA system rather than the primary production model for grid completion.

How to Get Started

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("uniba/cruciverb-it-IT5-full")
model = AutoModelForSeq2SeqLM.from_pretrained("uniba/cruciverb-it-IT5-full")

# Example input: "Un passo indietro nel tempo" (Target: ieri, length 4)
input_text = "Trova la soluzione. Lunghezza soluzione: 4. Indizio: Un passo indietro nel tempo"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Acknowledgments

We acknowledge the support of the PNRR project FAIR - Future AI Research (PE00000013), Spoke 6 - Symbiotic AI (CUP H97G22000210007) under the NRRP MUR program funded by the NextGenerationEU.

---

license: cc-by-nc-4.0