Drift-8B-Material

github blog arxiv

This repository contains a merged HuggingFace checkpoint fine-tuned based on Qwen/Qwen3-8B.

Quick Links

Model Summary

  • Base model: Qwen/Qwen3-8B
  • Architecture: Qwen3ForCausalLM
  • Precision: bfloat16
  • Context length (config): max_position_embeddings = 40960
  • Weights format: sharded safetensors (4 shards)

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Yiwei6534/Drift-8B-Material"

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What can you help me with?"},
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Generation Defaults

The bundled generation_config.json uses temperature=0.6, top_k=20, top_p=0.95. Adjust based on your deployment.

Integrity Files

  • FILE_MANIFEST.json: list of distributed files and their byte sizes.
  • SHA256SUMS.txt: SHA256 checksums for all distributed files (verify with sha256sum -c SHA256SUMS.txt).

Limitations

  • The model may hallucinate tool calls or produce invalid arguments.
  • Output quality depends on the serving template and tool schema formatting.
  • Safety, bias, and domain-specific failure modes are not fully documented here.

Citation

If you find DRIFT or this model helpful in your research, please cite:

@article{luo2026drift,
  title={DRIFT: Difficulty Routing Self-DIstillation with Rhythm-Gated Exploration and Success BuFfer Training},
  author={Luo, Haisen and Liu, Yiwei and Wang, Haoning and Liu, Dan and Yin, Junxi and Wang, Haotian and Zhang, Lei and Tian, Xiaoyu and Chen, Shuaiting and Song, Yuansheng and others},
  journal={arXiv preprint arXiv:2606.30345},
  year={2026}
}

License

This repository uses license: other as a placeholder. Replace it with the correct license for the base model, your fine-tuning data, and your distribution terms before publishing.

Downloads last month
17
Safetensors
Model size
8B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for linglingdan/DRIFT-8B-Material

Finetuned
Qwen/Qwen3-8B
Finetuned
(2,342)
this model

Paper for linglingdan/DRIFT-8B-Material