metadata
license: other
base_model: Qwen/Qwen3-4B
tags:
- clinvar
- acmg
- variant-classification
- dual-head
- sentence-classification
- text-classification
LLM4Variants-Qwen3-4B
Dual-head sentence classifier for ACMG evidence-code + strength prediction on ClinVar submission comments. This is rank #1 of the grid search (ranked by joint test accuracy).
The model wraps the backbone Qwen/Qwen3-4B with two heads on top of mean-pooled
hidden states:
- code head — 28-way ACMG evidence code (
PVS1, PS1–PS4, PM1–PM6, PP1–PP5, BA1, BS1–BS4, BP1–BP7+NO_KEYWORD) - strength head — 6-way strength, conditioned on a learned embedding of the
predicted code (
Supporting, Moderate, Strong, VeryStrong, NotMet, NoStrength)
Test metrics
| Metric | Value |
|---|---|
| Code accuracy | 0.9309 |
| Strength accuracy | 0.9374 |
| Joint accuracy | 0.8822 |
| Strength acc | correct code | 0.9477 |
| Code weighted-F1 | 0.9307 |
| Strength weighted-F1 | 0.9351 |
Training configuration
| Hyperparameter | Value |
|---|---|
| Learning rate | 0.0001 |
| Effective batch size | 128 |
| Epochs | 8 |
| Max length | 256 |
| λ (strength loss) | 1.0 |
| Code emb dim | 64 |
| Negative ratio | 0.25 |
| Seed | 42 |
| Train / val / test size | 19161 / 1278 / 5110 |
Files
model.safetensors— full state dict (backbone +code_head+code_embeddings+strength_head).label_mappings.json—keyword2id/strength2id(and reverse).- tokenizer files +
chat_template.jinja.
Loading
This is a custom nn.Module (DualHeadLLM), not a transformers
AutoModel. Reconstruct the module (see train_dual_head.py), then load the
weights:
from safetensors.torch import load_file
from huggingface_hub import hf_hub_download
model = DualHeadLLM("Qwen/Qwen3-4B", num_keywords=28, num_strengths=6)
state = load_file(hf_hub_download("HFXM/LLM4Variants-Qwen3-4B", "model.safetensors"))
model.load_state_dict(state)