README.md

# NEXS LoRA Adapters — Serving & Evaluation Guide

Rank-128 LoRA adapters extracted with
[mergekit](https://github.com/arcee-ai/mergekit) from domain fine-tunes and
sanitized for vLLM serving (pure `lora_A`/`lora_B` weights only;
`modules_to_save` tensors stripped, `modules_to_save: null` in configs; no
`resize_token_embeddings()` needed). Two families:

- **Llama-3.1-8B** — five adapters (finance, legal, medical, toxicity,
  truthfulness), 224 A/B pairs each (32 layers × q/k/v/o/gate/up/down).
- **Qwen3-32B** — six adapters (instruction-following, medical ×2,
  theorem proving, Russian ×2), 448 A/B pairs each (64 layers × 7 projections).

## Llama-3.1-8B adapters on the Hub

| Domain | HF repo | Extracted from |
|---|---|---|
| finance | [anjohn0077/NEXS-finance-lora](https://huggingface.co/anjohn0077/NEXS-finance-lora) | mukaj/Llama-3.1-Hawkish-8B |
| legal | [anjohn0077/NEXS-legal-lora](https://huggingface.co/anjohn0077/NEXS-legal-lora) | MaziyarPanahi/calme-2.3-legalkit-8b |
| medical | [anjohn0077/NEXS-medical-lora](https://huggingface.co/anjohn0077/NEXS-medical-lora) | TsinghuaC3I/Llama-3.1-8B-UltraMedical |
| toxicity | [anjohn0077/NEXS-toxicity-lora](https://huggingface.co/anjohn0077/NEXS-toxicity-lora) | K-intelligence/Llama-SafetyGuard-Content-Binary |
| truthfulness | [anjohn0077/NEXS-truthfulness-lora](https://huggingface.co/anjohn0077/NEXS-truthfulness-lora) | HiTZ/Llama-3.1-8B-Instruct-multi-truth-judge |

Index/manifest repo: [anjohn0077/NEXS-lora-adapters](https://huggingface.co/anjohn0077/NEXS-lora-adapters)

## Qwen3-32B adapters on the Hub

| Name | HF repo | Extracted from | Notes |
|---|---|---|---|
| IF | [anjohn0077/NEXS-qwen3-32b-IF-lora](https://huggingface.co/anjohn0077/NEXS-qwen3-32b-IF-lora) | qihoo360/Light-IF-32B | instruction following |
| medical-openmedzoo | [anjohn0077/NEXS-qwen3-32b-medical-openmedzoo-lora](https://huggingface.co/anjohn0077/NEXS-qwen3-32b-medical-openmedzoo-lora) | OpenMedZoo/MedGo | |
| medical-tachyhealth | [anjohn0077/NEXS-qwen3-32b-medical-tachyhealth-lora](https://huggingface.co/anjohn0077/NEXS-qwen3-32b-medical-tachyhealth-lora) | TachyHealth/Gazal-R1-32B-sft-merged-preview | fp32 tensors (4.3 GB) |
| prover | [anjohn0077/NEXS-qwen3-32b-prover-lora](https://huggingface.co/anjohn0077/NEXS-qwen3-32b-prover-lora) | Goedel-LM/Goedel-Prover-V2-32B | not yet benchmarked |
| russian-refalmachine | [anjohn0077/NEXS-qwen3-32b-russian-refalmachine-lora](https://huggingface.co/anjohn0077/NEXS-qwen3-32b-russian-refalmachine-lora) | RefalMachine/RuadaptQwen3-32B-Instruct | |
| russian-t-tech | [anjohn0077/NEXS-qwen3-32b-russian-t-tech-lora](https://huggingface.co/anjohn0077/NEXS-qwen3-32b-russian-t-tech-lora) | t-tech/T-pro-it-2.0 | |

## Serve the five Llama adapters with vLLM

Directly from the Hub (vLLM downloads the adapters itself; requires
`hf auth login` or `HF_TOKEN` for the gated base model):

```bash
python -m vllm.entrypoints.openai.api_server \
    --model meta-llama/Llama-3.1-8B \
    --enable-lora \
    --lora-modules \
        finance=anjohn0077/NEXS-finance-lora \
        legal=anjohn0077/NEXS-legal-lora \
        medical=anjohn0077/NEXS-medical-lora \
        toxicity=anjohn0077/NEXS-toxicity-lora \
        truthfulness=anjohn0077/NEXS-truthfulness-lora \
    --port 8000 \
    --max-loras 5 \
    --max-lora-rank 128 \
    --gpu-memory-utilization 0.85
```

Or from the local sanitized copies (on this cluster):

```bash
python -m vllm.entrypoints.openai.api_server \
    --model meta-llama/Llama-3.1-8B \
    --enable-lora \
    --lora-modules \
        finance=/scratch/shared_dir/lora/llama/finance_sanitized \
        legal=/scratch/shared_dir/lora/llama/legal_sanitized \
        medical=/scratch/shared_dir/lora/llama/medical_sanitized \
        toxicity=/scratch/shared_dir/lora/llama/toxicity_sanitized \
        truthfulness=/scratch/shared_dir/lora/llama/truthfulness_sanitized \
    --port 8000 \
    --max-loras 5 \
    --max-lora-rank 128 \
    --gpu-memory-utilization 0.85
```

To fetch the Hub copies to a local directory instead:

```bash
for d in finance legal medical toxicity truthfulness; do
    hf download "anjohn0077/NEXS-${d}-lora" --local-dir "./adapters/${d}"
done
```

## Serve the six Qwen3-32B adapters with vLLM

The base model is ~65 GB in bf16; one 141 GB H200 fits the model, all six
adapters, and KV cache at `--gpu-memory-utilization 0.85`.

```bash
python -m vllm.entrypoints.openai.api_server \
    --model Qwen/Qwen3-32B \
    --enable-lora \
    --lora-modules \
        IF=anjohn0077/NEXS-qwen3-32b-IF-lora \
        medical_openmedzoo=anjohn0077/NEXS-qwen3-32b-medical-openmedzoo-lora \
        medical_tachyhealth=anjohn0077/NEXS-qwen3-32b-medical-tachyhealth-lora \
        prover=anjohn0077/NEXS-qwen3-32b-prover-lora \
        russian_refalmachine=anjohn0077/NEXS-qwen3-32b-russian-refalmachine-lora \
        russian_t_tech=anjohn0077/NEXS-qwen3-32b-russian-t-tech-lora \
    --port 8000 \
    --max-loras 6 \
    --max-lora-rank 128 \
    --gpu-memory-utilization 0.85
```

Or from the local sanitized copies (on this cluster), replace each repo id
with `/scratch/shared_dir/lora/qwen/<dir>_sanitized` (dirs: `IF_sanitized`,
`medical_OpenMedZoo_sanitized`, `medical_TachyHealth_sanitized`,
`prover_sanitized`, `russian_RefalMachine_sanitized`,
`russian_t-tech_sanitized`).

## Evaluate with lm-evaluation-harness

With a server running on port 8000, each adapter is addressed by the name
given in `--lora-modules`. Llama family:

```bash
lm_eval --model local-completions \
    --model_args model=finance,base_url=http://localhost:8000/v1/completions,tokenizer=meta-llama/Llama-3.1-8B,num_concurrent=10 \
    --tasks mmlu_econometrics \
    --output_path results/vllm_finance

lm_eval --model local-completions \
    --model_args model=legal,base_url=http://localhost:8000/v1/completions,tokenizer=meta-llama/Llama-3.1-8B,num_concurrent=10 \
    --tasks mmlu_professional_law \
    --output_path results/vllm_legal

lm_eval --model local-completions \
    --model_args model=medical,base_url=http://localhost:8000/v1/completions,tokenizer=meta-llama/Llama-3.1-8B,num_concurrent=10 \
    --tasks mmlu_professional_medicine \
    --output_path results/vllm_medical

lm_eval --model local-completions \
    --model_args model=toxicity,base_url=http://localhost:8000/v1/completions,tokenizer=meta-llama/Llama-3.1-8B,num_concurrent=10 \
    --tasks sst2 \
    --output_path results/vllm_toxicity_sst2

lm_eval --model local-completions \
    --model_args model=truthfulness,base_url=http://localhost:8000/v1/completions,tokenizer=meta-llama/Llama-3.1-8B,num_concurrent=10 \
    --tasks truthfulqa_mc2 \
    --output_path results/vllm_truthfulness
```

Qwen3-32B family (note `tokenizer=Qwen/Qwen3-32B`):

```bash
lm_eval --model local-completions \
    --model_args model=IF,base_url=http://localhost:8000/v1/completions,tokenizer=Qwen/Qwen3-32B,num_concurrent=10 \
    --tasks ifeval \
    --output_path results/vllm_qwen_IF

lm_eval --model local-completions \
    --model_args model=medical_openmedzoo,base_url=http://localhost:8000/v1/completions,tokenizer=Qwen/Qwen3-32B,num_concurrent=10 \
    --tasks medqa_4options \
    --output_path results/vllm_qwen_medical_openmedzoo

lm_eval --model local-completions \
    --model_args model=medical_tachyhealth,base_url=http://localhost:8000/v1/completions,tokenizer=Qwen/Qwen3-32B,num_concurrent=10 \
    --tasks medqa_4options \
    --output_path results/vllm_qwen_medical_tachyhealth

lm_eval --model local-completions \
    --model_args model=russian_refalmachine,base_url=http://localhost:8000/v1/completions,tokenizer=Qwen/Qwen3-32B,num_concurrent=10 \
    --tasks m_mmlu_ru \
    --output_path results/vllm_qwen_russian_refalmachine

lm_eval --model local-completions \
    --model_args model=russian_t_tech,base_url=http://localhost:8000/v1/completions,tokenizer=Qwen/Qwen3-32B,num_concurrent=10 \
    --tasks m_mmlu_ru \
    --output_path results/vllm_qwen_russian_t_tech
```

(The prover adapter's intended minif2f evaluation has not produced results
yet, so it ships unbenchmarked.)

## Benchmark results

Accuracy of each LoRA served on the base model via vLLM, compared against the
plain base model and the original full fine-tune it was extracted from.

Llama-3.1-8B family:

| Domain | Benchmark | Base | LoRA (vLLM) | Full fine-tune |
|---|---|---|---|---|
| finance | mmlu_econometrics | 0.4825 | 0.4561 | 0.5526 |
| legal | mmlu_professional_law | 0.4941 | 0.4915 | 0.4948 |
| medical | mmlu_professional_medicine | 0.7169 | 0.7059 | 0.7721 |
| toxicity | sst2 | 0.6732 | 0.8991 | 0.8899 |
| truthfulness | truthfulqa_mc2 | 0.4416 | 0.7164 | 0.7307 |

Note: toxicity and truthfulness recover nearly all (or more than) the
fine-tune's gains; finance/medical/legal LoRAs land close to base — most of
those fine-tunes' improvement lived in the full-rank embedding/head deltas
that vLLM cannot serve and were stripped during sanitization.

Qwen3-32B family (from `qwen32b_adapter_acc.csv`):

| Adapter | Benchmark | Base | LoRA (vLLM) | Full fine-tune |
|---|---|---|---|---|
| IF | ifeval | 0.8336 | 0.2737 | 0.8669 |
| medical-openmedzoo | medqa_4options | 0.7494 | 0.7604 | 0.7596 |
| medical-tachyhealth | medqa_4options | 0.7494 | 0.7455 | 0.7478 |
| russian-refalmachine | m_mmlu_ru | 0.7517 | 0.7524 | 0.7370 |
| russian-t-tech | m_mmlu_ru | 0.7517 | 0.7542 | 0.7679 |
| prover | minif2f | — | — | — |

Notes: the IF LoRA scores far below base on ifeval (0.2737 vs 0.8336) — the
low-rank approximation appears to break the fine-tune's instruction-following
behavior rather than approximate it; it is published for completeness with
these honest numbers. The medical and russian adapters track their full
fine-tunes closely. prover has not been benchmarked (its minif2f run produced
no results), but it passes structural validation and serves through vLLM.

## Dependencies (for replication)

All extraction, sanitization, serving, and evaluation ran in a single Python
virtualenv on Linux with NVIDIA H200 GPUs (driver 595.71.05). Exact versions:

| Component | Version | Notes |
|---|---|---|
| Python | 3.12.3 | |
| torch | 2.9.1 | CUDA 12.8 build (`2.9.1+cu128`) |
| mergekit | 0.1.4 | **fork**: [ikhyunAn/mergekit](https://github.com/ikhyunAn/mergekit) @ `e85a454`, editable install — not upstream arcee-ai/mergekit |
| vllm | 0.15.1 | |
| transformers | 4.57.1 | |
| peft | 0.15.2 | |
| huggingface-hub | 0.35.3 | pinned `<1.0` (required by transformers 4.57.1) |
| safetensors | 0.5.3 | |
| accelerate | 1.6.0 | |
| tokenizers | 0.22.1 | |
| xformers | 0.0.32.post1 | |
| numpy | 2.2.6 | |
| lm_eval | 0.4.9.1 | [EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) @ `ad3f4d0c`, editable install |

To recreate the core environment:

```bash
python3.12 -m venv lora-env && source lora-env/bin/activate
pip install torch==2.9.1 --index-url https://download.pytorch.org/whl/cu128
pip install vllm==0.15.1 transformers==4.57.1 peft==0.15.2 \
    "huggingface_hub==0.35.3" safetensors==0.5.3 accelerate==1.6.0
pip install -e "git+https://github.com/ikhyunAn/mergekit.git@e85a454#egg=mergekit"
pip install -e "git+https://github.com/EleutherAI/lm-evaluation-harness.git@ad3f4d0c#egg=lm_eval"
```

Caveats:

- Do **not** upgrade `huggingface_hub` past 1.0 in this environment —
  transformers 4.57.1 pins `huggingface-hub<1.0` and both transformers and
  mergekit break at import time with 1.x.
- Extraction used a single GPU (`CUDA_VISIBLE_DEVICES=<idx>` with
  `mergekit-extract-lora --cuda`); each 8B extraction takes ~4 minutes on an
  H200 and peaks around 2×16 GB of downloaded weights in the HF cache.

## Provenance / pipeline

1. Extraction: `mergekit-extract-lora --model <fine-tune> --base-model
   <base> --out-path <dir> --cuda` with base `meta-llama/Llama-3.1-8B` for the
   llama family (see `extract_and_upload_loras.sh`; raw outputs in
   `/scratch/shared_dir/lora/llama/<domain>/`) and `Qwen/Qwen3-32B` for the
   qwen family (raw outputs in `/scratch/shared_dir/lora/qwen/`).
2. Sanitization (same script set in `/scratch/shared_dir/lora/llama/` and
   `/scratch/shared_dir/lora/qwen/`): `fix_adapters.py` (drop norm layers) →
   `fix_vllm_weights.py` / `final_clean.py` (drop all non-`lora_*` keys, null
   out `modules_to_save`) → `verify_all_clean.py`. For llama, toxicity's
   extended vocab (128258) was truncated to the base 128256 first via
   `fix_toxicity_vocab.py`.
3. Upload: `upload_sanitized_loras.py` (llama) and `upload_qwen_loras.py`
   (qwen), both in this directory, push each sanitized adapter to its Hub
   repo with an updated README.