environment.md

Environment Requirements

Hardware

• 8× NVIDIA H100 80GB (or A100 80GB) per training run
• ~500 GB shared disk for model weights + checkpoints + train data
• ~200 GB system RAM recommended (DeepSpeed ZeRO-3 large weights gathering)

OS

• Ubuntu 22.04 LTS (kernel 5.15+)
• CUDA 12.8 drivers (12.4+ should also work)

Three conda environments

We use THREE separate environments to avoid dependency conflicts between training (LlamaFactory + DeepSpeed) and inference (vLLM), plus a tiny CPU-only env for the ensemble. Each phase has a pinned requirements/<phase>.txt.

Env 1: llama-qa35 (TRAINING + Phase 1 data builders)

Used by data_builders/*.py and llamafactory-cli train configs/*.yaml.

conda create -n llama-qa35 python=3.12 -y
conda activate llama-qa35
pip install -r requirements/train.txt
# LlamaFactory installed in dev mode (git clone)
git clone https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory && pip install -e ".[deepspeed]"

Env 2: vllm (INFERENCE — Phase 3)

Used by scripts/launch_all_predicts.sh → 19 LoRA cand CSVs.

conda create -n vllm python=3.12 -y
conda activate vllm
pip install -r requirements/infer.txt

Env 3: rouge (ENSEMBLE — Phase 4, CPU only, any python 3.10+)

Used by scripts/build_ensemble.py to write the regenerated submission CSV.

conda create -n rouge python=3.12 -y          # or use any base python
conda activate rouge
pip install -r requirements/ensemble.txt

Workspace layout

Every script, YAML, and builder in this package is hard-coded to the workspace root /mnt/msrh/Magic_submission/. Extract the archive (or symlink) so the tree lives at that exact location. Inside it you also need:

/mnt/msrh/Magic_submission/
├── data/                                  # Zindi competition CSVs (Train/Val/Test/SampleSubmission)
├── hub/                                   # base + retrieval HF snapshots (see below)
│   ├── Qwen3.5-27B/
│   ├── Qwen3.6-27B/
│   ├── Qwen3-32B/
│   └── AfriE5-Large-instruct/
├── LF/data/                               # generated train/test JSONLs (created by data_builders/)
├── hf_cache/                              # HF cache for retrieval encoder
├── checkpoints/                           # 19 LoRA adapters shipped with this package
├── checkpoints_trained/                   # output dir for any from-scratch retrains
└── candidate_csvs/                        # generated per-cand prediction CSVs

Base models (HuggingFace)

Download into /mnt/msrh/Magic_submission/hub/:

HUB=/mnt/msrh/Magic_submission/hub

# Qwen3.5-27B-Base — main model
huggingface-cli download Qwen/Qwen3.5-27B --local-dir $HUB/Qwen3.5-27B

# Qwen3.6-27B-Base
huggingface-cli download Qwen/Qwen3.6-27B --local-dir $HUB/Qwen3.6-27B

# Qwen3-32B-Base
huggingface-cli download Qwen/Qwen3-32B --local-dir $HUB/Qwen3-32B

Retrieval model

# Multilingual E5 instruct (AfriE5) — used by all data_builders/*.py
huggingface-cli download McGill-NLP/AfriE5-Large-instruct \
  --local-dir /mnt/msrh/Magic_submission/hub/AfriE5-Large-instruct

Key packages cheatsheet

Package	Train env	Infer env	Notes
torch	2.11.0+cu128	2.10.0+cu128	DIFFERENT versions — keep envs separate
transformers	5.2.0	4.57.6	DIFFERENT — train needs newer for Qwen3.5/6
vllm	—	0.19.1	Inference only
llamafactory	0.9.5.dev0	—	Train only (editable install)
peft	0.18.1	0.19.1	LoRA support
deepspeed	0.18.8	—	ZeRO-3
rouge-score	—	— (CPU env)	0.1.2 for ensemble medoid_ngram