MODELS.md

Models

Detail for the 21 checkpoints. The system description and results are in the model card, the combiner in stacking_ensemble/, and the generated data in data/.

Encoders share one recipe, recorded in full in each model's best.json: max_len 128, batch 16, lr 2e-5, weight decay 0.01, seed 42, 12 epochs with patience 3, cross-entropy unless noted. Label order is ["Against","Favor","None"], so id2label is 0=Against, 1=Favor, 2=None. prep_mode: preserve keeps hashtags and elongation, strip removes them. Track-2 models prepend a short target description to help cross-target transfer.

LoRA adapters share r=16, α=32 over all 7 projection matrices (q, k, v, o, gate, up, down), with inference scoring the log-probability of the label continuation.

Dev F is the best dev F_avg2 reached during training. For Track 1 that is in-domain; for Track 2 it is the held-out Women Empowerment target. Neither is comparable to the test scores in the model card.

Encoders

Load with AutoModelForSequenceClassification.from_pretrained(repo, subfolder="models/encoders/<name>").

Subfolder	Base	Dev F	Ep.	Role
t1_marbert	UBC-NLP/MARBERTv2	0.8360	3	main Track-1 encoder, also the retrieval embedder
t1_arabert	aubmindlab/bert-base-arabertv02-twitter	0.8351	2	ensemble member
t1_arabert_strip	aubmindlab/bert-base-arabertv02-twitter	0.8385	7	preprocessing variant (strip)
t1_marbert_strip	UBC-NLP/MARBERTv2	0.8311	6	preprocessing variant (strip)
t1_camelbert	CAMeL-Lab/bert-base-arabic-camelbert-mix	0.7904	3	alternative family
t1_aramodern	NAMAA-Space/AraModernBert-Base-V1.0	0.7897	10	alternative family
t2_marbert	UBC-NLP/MARBERTv2	0.8195	2	main Track-2 encoder and retrieval embedder
t2_marbert_strip	UBC-NLP/MARBERTv2	0.8147	3	preprocessing variant (strip)
t2_marbert_weighted	UBC-NLP/MARBERTv2	0.8088	3	class-weighted loss, rejected
t2_marbert_aug	UBC-NLP/MARBERTv2	0.7930	3	trained on data/track2/train_aug.csv
t2_arabert	aubmindlab/bert-base-arabertv02-twitter	0.7837	3	ensemble member
t2_marbert_focal	UBC-NLP/MARBERTv2	0.7794	3	focal loss (γ=2), rejected
t2_camelbert	CAMeL-Lab/bert-base-arabic-camelbert-mix	0.7506	2	alternative family
t2_aramodern	NAMAA-Space/AraModernBert-Base-V1.0	0.7308	5	alternative family

MARBERTv2's lead over AraBERT-twitter is 0.1 points on Track 1 and 3.6 on Track 2. Both loss changes did worse than plain cross-entropy. This is experiment 1 in the model card.

LoRA adapters

Attach with PeftModel.from_pretrained(base, repo, subfolder="models/lora/<name>"). The base comes from its own repository.

Subfolder	Base	Trained on	Role
allam_t2	ALLaM-AI/ALLaM-7B-Instruct-preview	Track-2 train (Covid, Digital)	main Track-2 generative member, WE-dev Favg2 0.883, blended 0.5 with the few-shot member
allam_full	ALLaM-7B	Track-1 train	counterweight member; its Favor lean offsets the few-shot model's Against lean
allam_v2	ALLaM-7B	Track-1 train	second counterweight member
allam_loo	ALLaM-7B	Track-1 train, leave-one-target-out	transfer check
allam_real	ALLaM-7B	data/external/pool_real_all.csv	real-pool arm of the counterweight comparison
allam_style	ALLaM-7B	data/synth/t1_train_style.csv	style-pool arm; retraining for accuracy cost 0.0027 (experiment 7)
qwen_t2	Qwen/Qwen2.5-7B-Instruct	Track-2 train	decorrelation member, rejected at −0.0373 (experiment 7)

Notes

• Base models are referenced, not re-hosted. ALLaM-7B, Qwen2.5-7B and the served models (Gemma-4-31B, Qwen3.6-35B) are third party; only our adapters and fine-tuned encoders are here.
• Four encoders used as probability sources were never saved: AraELECTRA, XLM-R-large, ARBERTv2 and AraBERT-large. Their outputs are in stacking_ensemble/probs/, and code/ rebuilds them by overriding the base id (see code/README.md).
• The Qwen3.6-35B NVFP4 checkpoint needed an lm_head dequantization patch to serve. Gemma-4-31B NVFP4 was the main few-shot backend.