yhhugging/DualMLC-wiki10-31k

DualMLC — Wiki10-31K

Dataset-specific checkpoint for LLM-Enhanced Dual-Branch Learning for Large-Scale Multi-Label Text Classification by Hui Ye, Jing Zhang, Xiulong Yang, and Rajshekhar Sunderraman.

• Paper: https://arxiv.org/abs/2609.12915
• Code: https://github.com/huiyegit/DualMLC
• Paper page: https://huggingface.co/papers/2609.12915

DualMLC processes a document with a Qwen2.5-7B branch adapted using LoRA and a fully fine-tuned BERT branch. Each branch has its own classifier head. The final label ranking uses late logit fusion; the architecture, label count, and fusion weight are recorded in config.json.

The released adapter records a local Qwen2.5-7B-Instruct base model path. This release uses Qwen/Qwen2.5-7B-Instruct to match that record, even though the general GitHub setup instructions name the non-Instruct base.

The original checkpoint omitted the top-level architecture file. This release includes a reconstructed config.json: layer aggregation (last four layers) and pooling (Qwen mean, BERT CLS) were confirmed by the author; other values were recovered from the adapter, training log, and head shapes. See config_provenance.json and training.log. The logged Qwen head dropout is 0.2.

Checkpoint contents

config.json
qwen_lora_adapter/      # Qwen LoRA adapter; not the full 7B base model
bert_encoder/          # full fine-tuned BERT encoder
qwen_tokenizer/
bert_tokenizer/
head_qwen.pt
head_bert.pt
best_metrics.json      # if included in the original release
labels.txt             # optional label names, in training label-index order

The Qwen base model is downloaded separately from Qwen/Qwen2.5-7B-Instruct. Use the DualMLC loader below to reconstruct both branches and classifier heads.

Download and predict

Create the environment following the GitHub setup instructions, then run from the source repository:

git clone https://github.com/huiyegit/DualMLC.git
cd DualMLC
git checkout de43c6b11a85b6d641ae8dc468930df40dac96bd
pip install huggingface_hub

Download the checkpoint:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="yhhugging/DualMLC-wiki10-31k",
    local_dir="models/dualmlc-wiki10-31k",
)

Predict on a document:

python test.py \
    --ckpt models/dualmlc-wiki10-31k \
    --qwen-name Qwen/Qwen2.5-7B-Instruct \
    --no-eval \
    --text "A document to classify." \
    --predict-topk 5

Predictions are label IDs in this checkpoint's training label order. If you have the corresponding label vocabulary, pass --label-file path/to/labels.txt, with one label name per line. Label IDs are not interchangeable across datasets or dataset preprocessing variants.

To use the existing Python loader:

from model import load_checkpoint

model, qwen_tokenizer, bert_tokenizer, arch = load_checkpoint(
    "models/dualmlc-wiki10-31k",
    device="cuda",
    qwen_name="Qwen/Qwen2.5-7B-Instruct",
)

Evaluation

Dataset	P@1 (%)	P@3 (%)	P@5 (%)
Wiki10-31K	90.78	81.11	71.94

These values are recorded in the released checkpoint's best_metrics.json at step 4000 and agree with the GitHub Model Zoo after rounding. They are not a new evaluation of the uploaded files. The reported training time is 0.25 hours on 8 × RTX 4090 with CUDA 11.8. Refer to the source repository for the dataset preparation and dataset-specific training command.

To evaluate, download the matching preprocessed dataset as described in the source repository, then run:

python test.py \
    --ckpt models/dualmlc-wiki10-31k \
    --qwen-name Qwen/Qwen2.5-7B-Instruct \
    --data-dir xmc-base/wiki10-31k

Intended use and limitations

This checkpoint ranks labels from the fixed Wiki10-31K label vocabulary for multi-label text classification research. It does not generate text or accept an arbitrary new label vocabulary. Inputs are truncated to the maximum lengths in config.json; performance on other domains and languages has not been established by these reported results. A GPU with sufficient memory for the Qwen base model and both branches is recommended. The existing inference code reports sigmoid-transformed scores; these are not documented as calibrated probabilities.

License

The publisher declares the checkpoint license as apache-2.0. The source repository's Apache 2.0 license originally covers code only. The base models and benchmark datasets have their own licenses and terms; see Qwen2.5-7B and BERT for upstream information.

Citation

@article{ye2026dualmlc,
  title={LLM-Enhanced Dual-Branch Learning for Large-Scale Multi-Label Text Classification},
  author={Ye, Hui and Zhang, Jing and Yang, Xiulong and Sunderraman, Rajshekhar},
  journal={arXiv preprint arXiv:2609.12915},
  year={2026}
}