Instructions to use UEC-InabaLab/CounselTrajLM-Qwen2.5-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use UEC-InabaLab/CounselTrajLM-Qwen2.5-7B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="UEC-InabaLab/CounselTrajLM-Qwen2.5-7B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("UEC-InabaLab/CounselTrajLM-Qwen2.5-7B") model = AutoModelForCausalLM.from_pretrained("UEC-InabaLab/CounselTrajLM-Qwen2.5-7B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use UEC-InabaLab/CounselTrajLM-Qwen2.5-7B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "UEC-InabaLab/CounselTrajLM-Qwen2.5-7B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "UEC-InabaLab/CounselTrajLM-Qwen2.5-7B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/UEC-InabaLab/CounselTrajLM-Qwen2.5-7B
- SGLang
How to use UEC-InabaLab/CounselTrajLM-Qwen2.5-7B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "UEC-InabaLab/CounselTrajLM-Qwen2.5-7B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "UEC-InabaLab/CounselTrajLM-Qwen2.5-7B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "UEC-InabaLab/CounselTrajLM-Qwen2.5-7B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "UEC-InabaLab/CounselTrajLM-Qwen2.5-7B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use UEC-InabaLab/CounselTrajLM-Qwen2.5-7B with Docker Model Runner:
docker model run hf.co/UEC-InabaLab/CounselTrajLM-Qwen2.5-7B
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("UEC-InabaLab/CounselTrajLM-Qwen2.5-7B")
model = AutoModelForCausalLM.from_pretrained("UEC-InabaLab/CounselTrajLM-Qwen2.5-7B", device_map="auto")
messages = [
{"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))CounselTrajLM โ Qwen2.5-7B
๐ Paper: arXiv:2611.24168
CounselTrajLM is a Chinese counseling dialogue model fine-tuned on CounselTrajD to generate the next counselor response from dialogue history. This version uses Qwen2.5-7B-Instruct as its base.
This repository contains the complete merged BF16 model in nine safetensors shards, together with its tokenizer, chat template, inference script, and checksums. It can be loaded directly with Transformers.
Model and training details
| Item | Value |
|---|---|
| Base model | Qwen/Qwen2.5-7B-Instruct |
| Base revision | a09a35458c702b33eeacc393d103063234e8bc28 |
| Training data | CounselTrajD; dialogue-disjoint 90/10 train/validation split |
| Supervised training examples | 425,679 |
| Validation examples | 47,528 |
| Fine-tuning method | QLoRA; adapter merged into the unquantized BF16 base for release |
| LoRA rank / alpha / dropout | 32 / 64 / 0.05 |
| Learning rate | 1e-5 |
| Schedule / warmup ratio | Cosine / 0.06 |
| Epochs | 3; final-epoch export |
| Effective batch size | 16 |
| Maximum training sequence length | 2,048 tokens |
| Release format | Merged BF16 safetensors |
Validation was used for monitoring. The released model is the final epoch. See training_summary.json for the recorded settings.
Usage
The repository is public and can be downloaded without authentication. Install the dependencies in requirements.txt.
pip install -r requirements.txt
python generate_reply.py --messages examples/history.json
The script accepts a JSON list of chat messages with role and content keys.
Use user for client messages and assistant for counselor messages, ending with
a client message. It adds the training system instruction if none is supplied.
The example dialogue is newly written for demonstrating inference.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "UEC-InabaLab/CounselTrajLM-Qwen2.5-7B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id, dtype=torch.bfloat16, device_map="auto"
)
messages = [
{"role": "system", "content": "ไฝ ๆฏไธๅไธไธใๆธฉๅใ่ช็ถ็็บฟไธๅฟ็ๅจ่ฏขๅธใ่ฏทๆ นๆฎๅฏน่ฏๅๅฒ็ๆไธไธๅฅๅจ่ฏขๅธๅๅคใๅๅคๅชๅๅจ่ฏขๅธ่ฆ่ฏด็่ฏ๏ผไธ่ฆๅ่ฏด่ฏไบบๅ็งฐ๏ผไธ่ฆๅๅฏน่ฏๆ ็ญพใ"},
{"role": "user", "content": "ๆ่ฟๅทฅไฝไปปๅกๅพๅค๏ผไธ็ญไปฅๅไนไธ็ดๅจๆณๅทฅไฝ็ไบ๏ผๅพ้พๆพๆพใ"},
]
prompt = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.inference_mode():
output = model.generate(
**inputs, max_new_tokens=160, do_sample=True,
temperature=0.7, top_p=0.8, top_k=20, repetition_penalty=1.05,
)
print(tokenizer.decode(output[0, inputs.input_ids.shape[-1]:], skip_special_tokens=True))
The weights occupy approximately 15.2 GB, with additional memory required for inference. The release was checked with Transformers 5.13.0, PyTorch 2.5.1, Accelerate 1.13.0, and safetensors 0.8.0. Use SHA256SUMS and validation.json to check the release files and validation status.
Intended use and limitations
The model is intended for research on Chinese counseling dialogue generation. Its training data are synthetic. Generated responses may be inaccurate, inappropriate, or unsafe, and the model is not a substitute for professional mental health care.
The model is released under Apache-2.0; see LICENSE. The source data's licensing and attribution information is provided in the CounselTraj repository.
Citation
@inproceedings{qi-inaba-2027-counseltraj,
title = "{CounselTraj}: Synthesizing Process-Aware Psychological Counseling Dialogues via Dialogue-Act Trajectory Planning",
author = "Qi, Zhiyang and Inaba, Michimasa",
booktitle = "Proceedings of the 20th Conference of the European Chapter of the Association for Computational Linguistics",
year = "2027",
url = "https://arxiv.org/abs/2611.24168",
}
- Downloads last month
- 7
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="UEC-InabaLab/CounselTrajLM-Qwen2.5-7B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)