README.md

---
license: apache-2.0
language:
- ar
- en
tags:
- arabic
- genre-classification
- hierarchical-classification
- aragenre-2026
- arabicnlp-2026
- ensemble
model-index:
- name: aragenre-9model-ensemble-devtuned
  results:
  - task:
      type: text-classification
      name: Hierarchical Arabic Genre Classification
    dataset:
      type: aragenre-2026-dev
      name: AraGenre 2026 Development Set
    metrics:
    - type: hierarchical_macro_f1
      value: 0.9735
      name: Hierarchical Macro F1 (DEVELOPMENT SET, not a test-set metric)
---

# AraGenre 9-Model Ensemble — Dev-Tuned Weights

Extends the 8-model ensemble to 9 components by adding a synthetic-augmented E5-large model, for hierarchical Arabic genre classification. **This is not a single fine-tuned checkpoint** — the repo contains a weights JSON and a combination script, not trained weights of its own.

**Authors:** Hassan Barmandah (NAMAA Community; Umm Al-Qura University), Israa Elhosiny (NAMAA Community), Yousra El-Ghawi (NAMAA Community), Omer Nacar (NAMAA Community)

## ⚠️ Generalization Note

This ensemble's development-set score (0.9735 hierarchical F1) is **not** representative of real-world performance. Per the project's system-description paper, this entire lineage of fine-tuned/ensembled sentence encoders — which scored well on the 110-item, 6-genre AraGenre dev set — **collapsed to 0.22–0.44 hierarchical F1 on the actual 27,972-item hidden test set** (74 specific genres under 6 broad genres). Its component weights were fit via a dev-validated random search, so even the 0.9735 number reflects fitting to dev, not just evaluation on it.

The system that actually won for this team — **0.7013 hierarchical F1, 3rd of 18 teams** on the official CodaBench leaderboard — was a separate, zero-shot DeepSeek-LLM pipeline with no fine-tuning at all (`stage2_llm_zeroshot_pipeline/` in the project repo). This artifact is **not** that system. It is released here for transparency and reproducibility of the project's full experimental record, not as a recommended production classifier.

## Approach

Extends the 8-model ensemble to 9 components by adding the synthetic-augmented E5-large model. Since the exact original weights from an earlier run were never recorded, this artifact's weights are re-derived via a dev-validated random search (4,000 trials) over the weight simplex, using a Dirichlet prior seeded near the 8-model ensemble's known weights.

## Components

`bge-m3-zeroshot`, `bge-m3-augmented-defs`, `e5-large-cosine-8ep`, `e5-large-cosine-10ep`, `e5-large-mnrl-xgenre-lite`, `e5-large-mnrl-augmented-defs`, `e5-large-mnrl-xgenre-phase1`, `e5-large-multiseed-ensemble`, `e5-large-synthetic-augmented` (9 members).

## Training data

None directly — this is a weight recipe over pre-scored component models. The component weights were selected via a dev-validated random search against `dev_gold.json` labels.

## Usage

Requires cached dev score files from running all 9 component scripts first (`stage1_encoder_finetuning/scores/*_dev_scores.json`), then:

```bash
python ensemble_9model_devtuned.py
```

See the project repository for the full script and component-model requirements.

## Citation

If you use this work, please cite our system-description paper:

```bibtex
@inproceedings{barmandah-etal-2026-namaa,
  title = {NAMAA at AraGenre 2026: From Encoder Baselines to Self-Consistent LLM Ensembling for Hierarchical Arabic Genre Classification},
  author = {Barmandah, Hassan and Elhosiny, Israa and El-Ghawi, Yousra and Nacar, Omer},
  booktitle = {Proceedings of the 4th Arabic Natural Language Processing Conference (ArabicNLP 2026)},
  address = {Budapest, Hungary},
  publisher = {Association for Computational Linguistics},
  year = {2026},
}
```

Please also cite the AraGenre 2026 shared task overview paper:

```bibtex
@inproceedings{elhaj-etal-2026-aragenre,
  title = {AraGenre 2026: A Hierarchical Definition-Guided Arabic Genre Classification Shared Task},
  author = {El-Haj, Mo and Ezzini, Saad and Abudalfa, Shadi and Lamsiyah, Salima and Jarrar, Mustafa},
  booktitle = {Proceedings of the 4th Arabic Natural Language Processing Conference (ArabicNLP 2026)},
  address = {Budapest, Hungary},
  publisher = {Association for Computational Linguistics},
  year = {2026},
}
```
## License

Apache 2.0