| # Models |
|
|
| Detail for the 21 checkpoints. The system description and results are in the |
| [model card](README.md), the combiner in [`stacking_ensemble/`](stacking_ensemble/README.md), and the |
| generated data in [`data/`](data/README.md). |
|
|
| Encoders share one recipe, recorded in full in each model's `best.json`: max_len 128, batch 16, |
| lr 2e-5, weight decay 0.01, seed 42, 12 epochs with patience 3, cross-entropy unless noted. Label |
| order is `["Against","Favor","None"]`, so `id2label` is 0=Against, 1=Favor, 2=None. `prep_mode: |
| preserve` keeps hashtags and elongation, `strip` removes them. Track-2 models prepend a short target |
| description to help cross-target transfer. |
|
|
| LoRA adapters share r=16, α=32 over all 7 projection matrices (q, k, v, o, gate, up, down), with |
| inference scoring the log-probability of the label continuation. |
|
|
| `Dev F` is the best dev *F*<sub>avg2</sub> reached during training. For Track 1 that is in-domain; for |
| Track 2 it is the held-out Women Empowerment target. Neither is comparable to the test scores in the |
| model card. |
|
|
| ## Encoders |
|
|
| Load with `AutoModelForSequenceClassification.from_pretrained(repo, subfolder="models/encoders/<name>")`. |
|
|
| | Subfolder | Base | Dev *F* | Ep. | Role | |
| |---|---|---:|---:|---| |
| | `t1_marbert` | UBC-NLP/MARBERTv2 | 0.8360 | 3 | main Track-1 encoder, also the retrieval embedder | |
| | `t1_arabert` | aubmindlab/bert-base-arabertv02-twitter | 0.8351 | 2 | ensemble member | |
| | `t1_arabert_strip` | aubmindlab/bert-base-arabertv02-twitter | 0.8385 | 7 | preprocessing variant (`strip`) | |
| | `t1_marbert_strip` | UBC-NLP/MARBERTv2 | 0.8311 | 6 | preprocessing variant (`strip`) | |
| | `t1_camelbert` | CAMeL-Lab/bert-base-arabic-camelbert-mix | 0.7904 | 3 | alternative family | |
| | `t1_aramodern` | NAMAA-Space/AraModernBert-Base-V1.0 | 0.7897 | 10 | alternative family | |
| | `t2_marbert` | UBC-NLP/MARBERTv2 | 0.8195 | 2 | main Track-2 encoder and retrieval embedder | |
| | `t2_marbert_strip` | UBC-NLP/MARBERTv2 | 0.8147 | 3 | preprocessing variant (`strip`) | |
| | `t2_marbert_weighted` | UBC-NLP/MARBERTv2 | 0.8088 | 3 | class-weighted loss, rejected | |
| | `t2_marbert_aug` | UBC-NLP/MARBERTv2 | 0.7930 | 3 | trained on `data/track2/train_aug.csv` | |
| | `t2_arabert` | aubmindlab/bert-base-arabertv02-twitter | 0.7837 | 3 | ensemble member | |
| | `t2_marbert_focal` | UBC-NLP/MARBERTv2 | 0.7794 | 3 | focal loss (γ=2), rejected | |
| | `t2_camelbert` | CAMeL-Lab/bert-base-arabic-camelbert-mix | 0.7506 | 2 | alternative family | |
| | `t2_aramodern` | NAMAA-Space/AraModernBert-Base-V1.0 | 0.7308 | 5 | alternative family | |
|
|
| MARBERTv2's lead over AraBERT-twitter is 0.1 points on Track 1 and 3.6 on Track 2. Both loss changes |
| did worse than plain cross-entropy. This is experiment 1 in the model card. |
|
|
| ## LoRA adapters |
|
|
| Attach with `PeftModel.from_pretrained(base, repo, subfolder="models/lora/<name>")`. The base comes |
| from its own repository. |
|
|
| | Subfolder | Base | Trained on | Role | |
| |---|---|---|---| |
| | `allam_t2` | ALLaM-AI/ALLaM-7B-Instruct-preview | Track-2 train (Covid, Digital) | main Track-2 generative member, WE-dev *F*<sub>avg2</sub> 0.883, blended 0.5 with the few-shot member | |
| | `allam_full` | ALLaM-7B | Track-1 train | counterweight member; its Favor lean offsets the few-shot model's Against lean | |
| | `allam_v2` | ALLaM-7B | Track-1 train | second counterweight member | |
| | `allam_loo` | ALLaM-7B | Track-1 train, leave-one-target-out | transfer check | |
| | `allam_real` | ALLaM-7B | `data/external/pool_real_all.csv` | real-pool arm of the counterweight comparison | |
| | `allam_style` | ALLaM-7B | `data/synth/t1_train_style.csv` | style-pool arm; retraining for accuracy cost 0.0027 (experiment 7) | |
| | `qwen_t2` | Qwen/Qwen2.5-7B-Instruct | Track-2 train | decorrelation member, rejected at −0.0373 (experiment 7) | |
|
|
| ## Notes |
|
|
| * Base models are referenced, not re-hosted. ALLaM-7B, Qwen2.5-7B and the served models (Gemma-4-31B, |
| Qwen3.6-35B) are third party; only our adapters and fine-tuned encoders are here. |
| * Four encoders used as probability sources were never saved: AraELECTRA, XLM-R-large, ARBERTv2 and |
| AraBERT-large. Their outputs are in `stacking_ensemble/probs/`, and `code/` rebuilds them by |
| overriding the base id (see [`code/README.md`](code/README.md)). |
| * The Qwen3.6-35B NVFP4 checkpoint needed an `lm_head` dequantization patch to serve. Gemma-4-31B |
| NVFP4 was the main few-shot backend. |
|
|
