Top 1 Private Solution - Magic Team
Multilingual Health QA in Low-Resource African Languages Challenge
Submitter: Magic
Best submission: go.csv (Zindi ID vtbP7bCH, submitted 21 Jun 17:12)
Public LB: 0.738783 · Private LB: Top 1 🏆
1. Solution overview
19-candidate ensemble built with V2 medoid_ngram consensus:
- Base: 12 fewshot LoRA adapters across Qwen3.5/Qwen3.6/Qwen3-32B × K=3,4,5,7
- v8 prompt swap: 1 Qwen3.5-27B K=5 LoRA trained with anchored-extraction prompt
- 3 cross-arch "less-overfit" checkpoints
- 3 no-fewshot mediators
Per Test row, the medoid (highest sum of pairwise ROUGE-1.F + ROUGE-2.F) is chosen as the final answer.
Why it wins
Six independent decorrelation axes simultaneously:
- Architecture: Qwen3.5-27B + Qwen3.6-27B + Qwen3-32B (3 base models)
- K-count: K=3, K=4, K=5, K=7 fewshot demos
- Prompt recipe: v1 baseline vs v8 anchored-extraction
- Training schedule: 3ep vs 5ep
- Mediator: 3 no-fewshot baselines
2. Workspace layout
All paths in this package are hard-coded to /mnt/msrh/Magic_submission/. Extract this archive to that exact location (or mount/symlink it there) so every script — data builders, training YAMLs, inference launcher, ensemble — works without editing.
# One-time setup — choose ONE of these:
# (a) Extract the archive to /mnt/msrh/
sudo mkdir -p /mnt/msrh && sudo chown $USER /mnt/msrh
tar -xf Magic_submission.tar -C /mnt/msrh/ # extracts to /mnt/msrh/Magic_submission/
# (b) OR symlink an existing copy
ln -s /path/to/existing/Magic_submission /mnt/msrh/Magic_submission
# Verify
ls /mnt/msrh/Magic_submission/{go.csv,checkpoints,configs,scripts,data_builders}
After extraction, you must also place under /mnt/msrh/Magic_submission/:
data/Train.csv,data/Val.csv,data/Test.csv,data/SampleSubmission.csv(Zindi competition data — download separately). Or you can manually download it from the Zindi data source.hub/Qwen3.5-27B/,hub/Qwen3.6-27B/,hub/Qwen3-32B/(HF snapshot of each base model)hub/AfriE5-Large-instruct/(HF snapshot ofMcGill-NLP/AfriE5-Large-instruct)
/mnt/msrh/Magic_submission/
├── go.csv # Final submission CSV (== Zindi sub vtbP7bCH)
├── README.md # This file — full reproduction guide
├── environment.md # All env setup (conda, packages, models)
├── checkpoints/ # ⭐ 19 LoRA adapters (35 GB) — INPUT to inference
│ # Each folder = 1 LoRA adapter with:
│ # adapter_model.safetensors (1.8 GB for 27B, 2.1 GB for 32B)
│ # adapter_config.json
│ # chat_template.jinja
│ # tokenizer_config.json + tokenizer.json + processor_config.json
│ ├── Qwen3.5-27B-3fewshots-bs64-3eps-ckpt-1200/
│ ├── Qwen3.5-27B-3fewshots-bs64-3eps-ckpt-1100/
│ ├── Qwen3.5-27B-3fewshots-bs64-5eps-ckpt-1200/
│ ├── Qwen3.5-27B-4fewshots-bs64-3eps-ckpt-1600/
│ ├── Qwen3.5-27B-5fewshots-bs64-3eps-v8prompt-ckpt-1500/
│ ├── Qwen3.5-27B-7fewshots-bs64-3eps-ckpt-1600/
│ ├── Qwen3.5-27B-7fewshots-bs64-3eps-ckpt-1200/
│ ├── Qwen3.6-27B-3fewshots-bs64-3eps-ckpt-1600/
│ ├── Qwen3.6-27B-4fewshots-bs64-3eps-ckpt-1400/
│ ├── Qwen3.6-27B-5fewshots-bs64-3eps-ckpt-1200/
│ ├── Qwen3.6-27B-5fewshots-bs64-3eps-ckpt-1000/
│ ├── Qwen3.6-27B-7fewshots-bs64-3eps-ckpt-1600/
│ ├── Qwen3-32B-3fewshots-bs64-3eps-ckpt-1400/
│ ├── Qwen3-32B-5fewshots-bs64-3eps-ckpt-1700/
│ ├── Qwen3-32B-7fewshots-bs64-3eps-ckpt-1600/
│ ├── Qwen3-32B-7fewshots-bs64-3eps-ckpt-1200/
│ ├── Qwen3.5-27B-NoFewshots-bs64-5eps-ckpt-2800/
│ ├── Qwen3.6-27B-NoFewshots-bs64-5eps-ckpt-2600/
│ └── Qwen3-32B-NoFewshots-bs64-4eps-ckpt-6500/
├── candidate_csvs/ # OUTPUT folder for 19 per-model prediction CSVs
│ # NOTE: any CSVs here are reference/audit-trail copies.
│ # For verification, REGENERATE all 19 from checkpoints/ via launch_all_predicts.sh.
│ # The ensemble script (build_ensemble.py) reads CSVs from this folder.
├── LF/
│ └── data/
│ └── dataset_info.json
├── requirements/ # per-phase pip pinned deps
│ ├── train.txt # Phase 1 (data builders) + Phase 2 (SFT)
│ ├── infer.txt # Phase 3 (vLLM inference)
│ └── ensemble.txt # Phase 4 (CPU medoid ensemble)
├── data_builders/
│ ├── build_afrie5_k5.py
│ ├── build_fewshot_train.py
│ ├── build_fewshot_train_k4.py
│ ├── build_fewshot_train_k5.py
│ ├── build_fewshot_train_k7.py
│ ├── build_fewshot_test.py
│ ├── build_fewshot_test_k4.py
│ ├── build_fewshot_test_k5.py
│ └── build_fewshot_test_k7.py
├── configs/
│ # ── Fewshot SFT runs (13) ──
│ ├── qwen35_27b_fewshot_3ep_8gpu.yaml
│ ├── qwen35_27b_fewshot_k4_3ep_8gpu.yaml
│ ├── qwen35_27b_fewshot_k5_v8_3ep_8gpu.yaml
│ ├── qwen35_27b_fewshot_k7_3ep_8gpu.yaml
│ ├── qwen35_27b_fewshot_2ep_8gpu.yaml
│ ├── qwen36_27b_fewshot_k4_3ep_8gpu.yaml
│ ├── qwen36_27b_fewshot_k5_3ep_8gpu.yaml
│ ├── qwen36_27b_fewshot_k7_v8_3ep_8gpu.yaml # Q3.6 K=7 v8 prompt 3ep → ck-1600
│ ├── qwen3_32b_fewshot_3ep_bs64_r128_all_8gpu.yaml
│ ├── qwen3_32b_fewshot_3ep_r128_all_8gpu.yaml
│ ├── qwen3_32b_fewshot_k5_3ep_bs64_r128_all_8gpu.yaml
│ ├── qwen3_32b_fewshot_k7_3ep_bs64_r128_all_8gpu.yaml
│ # ── NoFewshots (mediator) SFT runs (3) ──
│ ├── qwen35_27b_nofewshots_5ep_bs64_8gpu.yaml
│ ├── qwen36_27b_nofewshots_5ep_bs64_8gpu.yaml
│ ├── qwen3_32b_nofewshots_4ep_bs64_8gpu.yaml
│ ├── fewshot_e3_Q36.sh
│ ├── ds_z3_config.json
├── scripts/ # Inference + ensemble scripts
│ ├── build_v8_k5_fewshot.py # v8 prompt swap (V1 → V8 prefix)
│ ├── vllm_predict_extra.py # vLLM batch inference w/ LoRA
│ ├── jsonl_rowidx_to_csv.py # jsonl → CSV converter
│ ├── rag_to_zindi.py # Alternate CSV builder (Plan D path)
│ ├── launch_all_predicts.sh # ⭐ Orchestrator: runs all 19 inferences → candidate_csvs/<descriptive>.csv
│ ├── build_ensemble.py # ⭐ FINAL ensemble medoid (uses candidate_csvs/)
│ └── ds_z3_config.json
└── docs/
└── (additional documentation if needed)
3. Verification flow (RECOMMENDED for reviewers — uses provided checkpoints)
The 19 LoRA adapters in checkpoints/ are the WINNING trained weights — the official artifacts that produced go.csv. Reviewers do NOT need to retrain. Use them directly to regenerate the 19 prediction CSVs, then run the ensemble.
All commands below run from /mnt/msrh/Magic_submission/ (paths are hard-coded — no editing required).
cd /mnt/msrh/Magic_submission
# Step 1: Setup envs (see environment.md for full conda setup)
conda activate llama-qa35 && pip install -r requirements/train.txt # Phase 1 + 2
conda activate vllm && pip install -r requirements/infer.txt # Phase 3
conda activate rouge && pip install -r requirements/ensemble.txt # Phase 4
# Step 2: Build TEST data (AfriE5 retrieval + fewshot demos) — train data not needed for verification
conda activate llama-qa35
python data_builders/build_afrie5_k5.py --k 3 # K=3 base retrieval (NoFewshots test JSON)
python data_builders/build_fewshot_test.py # K=3 fewshot test demos
python data_builders/build_fewshot_test_k4.py # K=4 fewshot test demos
python data_builders/build_fewshot_test_k5.py # K=5 fewshot test demos
python data_builders/build_fewshot_test_k7.py # K=7 fewshot test demos
# v8 variants — REQUIRE K=5 / K=7 fewshot files to already exist:
python scripts/build_v8_k5_fewshot.py # rebuilds *_k5_v8.json from K=5 fewshot
python scripts/build_v8_k7_fewshot.py # rebuilds *_k7_v8.json from K=7 fewshot
# Step 3: Generate 19 prediction CSVs from provided checkpoints (Reads checkpoints/, writes candidate_csvs/)
conda activate vllm
bash scripts/launch_all_predicts.sh
# Step 4: Run ensemble (V2 medoid_ngram) → final CSV
conda activate rouge
python scripts/build_ensemble.py
# Writes the regenerated submission CSV alongside the shipped go.csv.
# Reference md5 of shipped go.csv: a2ecca4a8e1aa01acf9a8b9a1d56ebf2
Note on byte-identity vs functional reproducibility
We E2E-verified this pipeline on a fresh 8×H100 box (vLLM 0.19.1 + torch 2.10+cu128).
Expect the regenerated CSV to NOT be md5-equal to the shipped go.csv — but the LB
will reproduce within ~0.005 public / ~0.001 private of the winning result.
| Run | Public LB | Private LB | Row match vs go.csv |
|---|---|---|---|
Original vtbP7bCH (shipped go.csv) |
0.738783 | 0.730865 | byte-identical (md5 a2ecca4a...) |
Re-run #1 — max_num_seqs=32 hard-coded |
0.734035 | 0.729336 | 65.0% |
Re-run #2 — per-cand max_num_seqs matched |
0.734104 | 0.729553 | 67.8% |
Root cause of the ~0.005 gap: vLLM 0.19.1 paged-attention has inherent non-determinism
between runs even at --temperature 0.0 with identical flags (CUDA atomic ops, bfloat16
rounding-order in attention, KV-cache block scheduling races). This affects ~30-40% of
generated tokens at single-cand level; the V2 medoid_ngram ensemble suppresses most of the
drift, leaving ~32% of per-row picks different but only a ~0.001 private-LB hit.
Acceptance criterion: if the regenerated CSV yields Private LB ≥ 0.728 (within −0.003 of the shipped 0.730865), reproduction is considered successful and the pipeline is verified. md5 byte-equality is NOT achievable on a different machine/run.
3b. Full from-scratch reproduction (optional — trains 13 LoRAs from base models)
Only needed if you want to retrain the LoRA adapters yourself instead of using the provided ones.
cd /mnt/msrh/Magic_submission
conda activate llama-qa35 # training env
# Build train data (same as Step 2 above but for train set)
python data_builders/build_afrie5_k5.py --k 3
python data_builders/build_afrie5_k5.py --k 5
python data_builders/build_afrie5_k5.py --k 7
python data_builders/build_fewshot_train.py
python data_builders/build_fewshot_train_k4.py
python data_builders/build_fewshot_train_k5.py
python data_builders/build_fewshot_train_k7.py
# v8 prompt swap — runs AFTER build_fewshot_train_k5.py and build_fewshot_test_k5.py
python scripts/build_v8_k5_fewshot.py # rebuilds *_k5_v8.json (train + test)
# Train each of the 16 LoRA adapters (configs/*.yaml). Outputs go to
# /mnt/msrh/Magic_submission/checkpoints_trained/<config-name>/ as set inside each YAML.
# Example for the key v8 model:
FORCE_TORCHRUN=1 NPROC_PER_NODE=8 \
llamafactory-cli train /mnt/msrh/Magic_submission/configs/qwen35_27b_fewshot_k5_v8_3ep_8gpu.yaml
# ... repeat for all 16 configs (~14h each on 8× H100)
# Replace adapters in checkpoints/<descriptive_name>/ with your fresh ckpts, then
# follow Step 3 + Step 4 from Section 3 above.
4. Data preparation (full pipeline)
Inputs (required)
- Zindi competition data:
Train.csv,Val.csv,Test.csv,SampleSubmission.csv(from competition page) - Path expected:
/mnt/msrh/Magic_submission/data/{Train,Val,Test,SampleSubmission}.csv(no editing needed — every builder and script reads from this fixed location)
Pipeline
AfriE5 base retrieval (
build_afrie5_k5.py)- For each train query, retrieve top-K from Train+Val pool using AfriE5 cosine similarity
- For each test query, retrieve top-K from Train+Val pool
- Output:
msrh_rag_train_afrie5_TV_k{K}.json+msrh_rag_test_k3_AfriE5_TV.json - Includes language-specific instruction tag (Akan/Amharic/Luganda/Swahili/English)
Fewshot demo prep (
build_fewshot_train_k{K}.py+build_fewshot_test_k{K}.py)- Per row, retrieve K AfriE5-similar same-subset (Q,A) pairs from Train+Val
- Prepend as "Example N:" demos in user message
- Output:
msrh_rag_train_afrie5_TV_k{K}_fewshot.json
v8 prompt swap (
build_v8_k5_fewshot.py)- Replace v1 prompt:
Use the retrieved contexts as your primary sources — copy exact phrasing where the contexts already address the question. Be concise and factually accurate. - With v8 prompt (33 words, 220 chars):
The retrieved contexts are your source of truth — copy or paraphrase their exact phrasing to answer. Reply in the same language and script as the question. Plain prose, no disclaimers or meta-commentary. - Output:
msrh_rag_train_afrie5_TV_k5_fewshot_v8.json
- Replace v1 prompt:
5. Training (13 LoRA adapters)
All training uses LlamaFactory + DeepSpeed ZeRO-3 on 8× H100 (80GB).
Common hyperparameters (all 13 runs)
finetuning_type: lora
lora_rank: 128
lora_alpha: 256
lora_dropout: 0.05
lora_target: all # all linear layers
learning_rate: 2.0e-4
lr_scheduler_type: cosine
warmup_ratio: 0.05
bf16: true
gradient_checkpointing: true
deepspeed: scripts/ds_z3_config.json
Per-model details
Per-model details (16 SFT runs total — all reproducible via configs/*.yaml)
The 19 ensemble candidates come from 16 distinct SFT training runs (3 of the runs export 2 checkpoints each as "early-tap" anti-overfit cands).
Fewshot runs (13 SFT, each prepends top-K AfriE5-similar (Q,A) demos)
| # | Model | K | Prompt | Epochs | cutoff_len | eff_bs | YAML | Best ckpt | Cand(s) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Qwen3.5-27B | 3 | v1 | 3 | 4096 | 64 | qwen35_27b_fewshot_3ep_8gpu.yaml | ck-1200 | ck-1200 + ck-1100 (early-tap) |
| 2 | Qwen3.5-27B | 3 | v1 | 5 | 4096 | 64 | qwen35_27b_fewshot_5ep_8gpu.yaml (modify epochs=5 in run 1's YAML) | ck-1200 | 1 cand |
| 3 | Qwen3.5-27B | 4 | v1 | 3 | 5120 | 64 | qwen35_27b_fewshot_k4_3ep_8gpu.yaml | ck-1600 | 1 cand |
| 4 | Qwen3.5-27B | 5 | v8 | 3 | 6144 | 64 | qwen35_27b_fewshot_k5_v8_3ep_8gpu.yaml | ck-1500 ⭐ | 1 cand ⭐ |
| 5 | Qwen3.5-27B | 7 | v1 | 3 | 8192 | 64 | qwen35_27b_fewshot_k7_3ep_8gpu.yaml | ck-1600 | ck-1600 + ck-1200 (early-tap) |
| 6 | Qwen3.6-27B | 3 | v1 (RecA) | 3 | 4096 | 64 | qwen36_27b_fewshot_3eps_bs64.yaml (Q3.5 K=3 YAML w/ model+template swap) | ck-1600 | 1 cand |
| 7 | Qwen3.6-27B | 4 | v1 | 3 | 5120 | 64 | qwen36_27b_fewshot_k4_3ep_8gpu.yaml | ck-1400 | 1 cand |
| 8 | Qwen3.6-27B | 5 | v1 | 3 | 6144 | 64 | qwen36_27b_fewshot_k5_3ep_8gpu.yaml | ck-1200 | ck-1200 + ck-1000 (early-tap) |
| 9 | Qwen3.6-27B | 7 | v8 | 3 | 8192 | 64 | qwen36_27b_fewshot_k7_v8_3ep_8gpu.yaml | ck-1600 | 1 cand |
| 10 | Qwen3-32B | 3 | v1 | 3 | 4096 | 64 | qwen3_32b_fewshot_3ep_bs64_r128_all_8gpu.yaml | ck-1400 | 1 cand |
| 11 | Qwen3-32B | 5 | v1 | 3 | 6144 | 64 | qwen3_32b_fewshot_k5_3ep_bs64_r128_all_8gpu.yaml | ck-1700 | 1 cand |
| 12 | Qwen3-32B | 7 | v1 | 3 | 8192 | 64 | qwen3_32b_fewshot_k7_3ep_bs64_r128_all_8gpu.yaml | ck-1600 | ck-1600 + ck-1200 (early-tap) |
NoFewshots runs (3 SFT, mediators — same K=3 AfriE5 retrieval but WITHOUT demos prepended)
These 3 are trained the same way as the fewshot runs but use the non-fewshot K=3 dataset (msrh_rag_train_afrie5_TV_k3.json — output of build_afrie5_k5.py --k 3 only, no build_fewshot_train*.py step). They serve as decorrelated "no-context-demos" mediators in the ensemble.
| # | Model | K | Prompt | Epochs | cutoff_len | eff_bs | YAML | Best ckpt | Cand |
|---|---|---|---|---|---|---|---|---|---|
| 13 | Qwen3.5-27B | 3 | v1 | 5 | 4096 | 64 | qwen35_27b_nofewshots_5ep_bs64_8gpu.yaml | ck-2800 | 1 cand |
| 14 | Qwen3.6-27B | 3 | v1 | 5 | 4096 | 64 | qwen36_27b_nofewshots_5ep_bs64_8gpu.yaml | ck-2600 | 1 cand |
| 15 | Qwen3-32B | 3 | v1 | 4 | 4096 | 64 | qwen3_32b_nofewshots_4ep_bs64_8gpu.yaml | ck-6500 | 1 cand |
Differences vs fewshot runs: only dataset: msrh_rag_train_afrie5_TV_k3 (vs *_k3_fewshot) and longer epoch budget (4-5 vs 3) since no-demo trains more slowly.
Total: 15 SFT runs → 19 ensemble cands (3 runs contribute 2 ckpts each)
Launch example (any config)
HF_DATASETS_OFFLINE=1 TRANSFORMERS_OFFLINE=1 \
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
FORCE_TORCHRUN=1 NNODES=1 NPROC_PER_NODE=8 \
llamafactory-cli train /mnt/msrh/Magic_submission/configs/qwen35_27b_fewshot_k5_v8_3ep_8gpu.yaml \
> /mnt/msrh/Magic_submission/training_logs/q35_k5_v8.log 2>&1
Save_steps=100-500 (depends on config), save_total_limit=25 (keep multiple ckpts for early-tap predictions).
Wall time per run (8× H100 80GB)
- Q3.5-27B 3ep: ~7-8h
- Q3.6-27B 3ep: ~8-10h
- Q3-32B 3ep: ~13-14h
- Q3-32B 4ep (NoFewshots): ~18h
- Q3.5/3.6-27B 5ep (NoFewshots): ~12-15h
- Total for 15 SFT runs (sequential): ~7-9 days
- With 2-3 nodes parallel: ~3-4 days
6. Inference (each LoRA adapter)
Single-GPU inference per adapter via vLLM (env: vllm). The wrapper launch_all_predicts.sh orchestrates all 19 runs and writes outputs into candidate_csvs/<descriptive_name>.csv ready for ensemble.
One-shot run all 19 predictions
bash /mnt/msrh/Magic_submission/scripts/launch_all_predicts.sh
# Outputs: /mnt/msrh/Magic_submission/candidate_csvs/{Qwen3.5-27B-..., Qwen3.6-27B-..., Qwen3-32B-...}.csv (19 files)
Manual single-cand example (the key v8 ckpt)
ROOT=/mnt/msrh/Magic_submission
CUDA_VISIBLE_DEVICES=0 python $ROOT/scripts/vllm_predict_extra.py \
--base $ROOT/hub/Qwen3.5-27B \
--adapter $ROOT/checkpoints/Qwen3.5-27B-5fewshots-bs64-3eps-v8prompt-ckpt-1500 \
--rag_test $ROOT/LF/data/msrh_rag_test_k3_AfriE5_TV_fewshot_k5_v8.json \
--out_jsonl $ROOT/_predict_workdir/Qwen3.5-27B-5fewshots-bs64-3eps-v8prompt-ckpt-1500/predictions.jsonl \
--max_lora_rank 128 --max_new 512 \
--max_model_len 8192 --mem_util 0.88 --max_num_seqs 32 \
--temperature 0.0 --top_p 1.0 --best_of 1 --no_think
python $ROOT/scripts/jsonl_rowidx_to_csv.py \
--jsonl $ROOT/_predict_workdir/Qwen3.5-27B-5fewshots-bs64-3eps-v8prompt-ckpt-1500/predictions.jsonl \
--out_csv $ROOT/candidate_csvs/Qwen3.5-27B-5fewshots-bs64-3eps-v8prompt-ckpt-1500.csv
Critical inference flags
--no_think— passesenable_thinking=Falseto chat template (Qwen3.5/6 trained withqwen3_5_nothinktemplate). Default ON wastes generation budget on reasoning; LB drops 0.10-0.25.--temperature 0.0 --best_of 1— greedy deterministic decoding--max_model_len: 6144 for K=3/4; 8192 for K=5; 10240 for K=7 (long prompts)
Provide --adapter checkpoints/<descriptive_name> pointing into the provided checkpoints/ folder. Run inference for all 19 candidates (each on a different GPU in parallel via the launcher). The output 19 CSVs go to candidate_csvs/, ready for ensemble.
Note: This package does NOT ship pre-computed
candidate_csvs/*.csv. They must be regenerated fromcheckpoints/viascripts/launch_all_predicts.shso the reviewer can independently verify the inference step.
7. Final ensemble (V2 medoid_ngram)
# (no GPU needed — pure CPU + ROUGE scoring)
pip install rouge-score==0.1.2
python scripts/build_ensemble.py
# Reads 19 CSVs from candidate_csvs/<descriptive_name>.csv
# Computes per-row medoid, writes the regenerated submission CSV
# Compares md5 against the shipped go.csv: a2ecca4a8e1aa01acf9a8b9a1d56ebf2
V2 medoid_ngram algorithm
For each Test row:
- Compute pairwise ROUGE-1.F + ROUGE-2.F between all 19 candidate answers
- For each candidate, sum its similarities to all OTHERS
- Pick the candidate with HIGHEST sum (the "medoid" of the cluster)
- Use its answer text as final prediction
Implementation (excerpt from build_ensemble.py):
from rouge_score import rouge_scorer
scorer = rouge_scorer.RougeScorer(["rouge1", "rouge2"], use_stemmer=False)
def medoid(texts):
best, best_s = 0, -1.0
for i in range(len(texts)):
s = sum(
scorer.score(texts[j], texts[i])["rouge1"].fmeasure
+ scorer.score(texts[j], texts[i])["rouge2"].fmeasure
for j in range(len(texts)) if i != j
)
if s > best_s: best_s, best = s, i
return best
8. Reproducibility
Hash of final submission
$ md5sum go.csv
a2ecca4a8e1aa01acf9a8b9a1d56ebf2 go.csv
Pinned versions (env)
See environment.md — exact torch / transformers / vllm / llamafactory versions.
Random seed
- LlamaFactory default seed (42) used for all trainings — reproducible LoRA weights given same env/data
- vLLM inference is deterministic with
--temperature 0.0 --best_of 1 - Ensemble medoid is purely deterministic (no random selection)
Sanity checks
- Verify CSV row order matches
data/Test.csv(2618 rows, 4 target columns: TargetRLF1, TargetR1F1, TargetLLMJudge, all = same answer) - Confirm md5sum of
go.csvmatches above
9. Acknowledgments
- AfriE5 (
McGill-NLP/AfriE5-Large-instruct) — multilingual retrieval encoder - LlamaFactory — LoRA training framework
- vLLM — inference engine (LoRA + batched generation)
- DeepSpeed — ZeRO-3 distributed training
- Qwen team (Alibaba) — Qwen3.5/3.6/Qwen3 base models
10. Contact
Submitter: Magic Submission deadline: 23 Jun 2026 17:00 GMT (handled in time ✓)