multicfe CDA
Collection
base and CDA cross- and multilingual mbert-based models trained on XNLI and sib200 • 16 items • Updated
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
XNLI CDA Model with Gemma
This model was trained on the XNLI dataset using Counterfactual Data Augmentation (CDA) with counterfactuals generated by Gemma.
Training Parameters
- Dataset: XNLI
- Mode: CDA
- Selection Model: Gemma
- Selection Method: Random
- Train Size: 2400 examples
- Epochs: 8
- Batch Size: 16
- Effective Batch Size: 64 (batch_size * gradient_accumulation_steps)
- Learning Rate: 1e-05
- Patience: 6
- Max Length: 256
- Gradient Accumulation Steps: 4
- Warmup Ratio: 0.1
- Weight Decay: 0.01
- Optimizer: AdamW
- Scheduler: cosine_with_warmup
- Random Seed: 42
Performance
- Overall Accuracy: 66.00%
- Overall Loss: 0.0133
Language-Specific Performance
- English (EN): 74.61%
- German (DE): 68.66%
- Arabic (AR): 65.11%
- Spanish (ES): 70.98%
- Hindi (HI): 61.10%
- Swahili (SW): 55.57%
Model Information
- Base Model: bert-base-multilingual-cased
- Task: Natural Language Inference
- Languages: 6 languages (EN, DE, AR, ES, HI, SW)
- Downloads last month
- -
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support