Rumiii/LWQwenMed_Human_Cognition

LWQwenMed — Human Cognition

Medical Chain-of-Thought Reasoning | Fine-Tuned on Qwen2.5-0.5B-Instruct

---

Overview

LWQwenMed_Human_Cognition is a supervised fine-tuned (SFT) variant of Qwen2.5-0.5B-Instruct, trained to produce structured, deliberate, step-by-step clinical reasoning in response to complex medical queries.

The model is trained to emulate the chain-of-thought (CoT) reasoning patterns characteristic of professional clinical analysis — working through symptoms, differential diagnoses, and pathophysiological mechanisms before arriving at a conclusion.

Fine-tuning was performed using the Unsloth framework with LoRA on a Tesla T4 GPU (Kaggle free tier), prioritizing memory efficiency without compromising reasoning depth.

---

Architecture & Training

Configuration

Parameter	Value
Base Model	Qwen2.5-0.5B-Instruct
Fine-tuning Method	Supervised Fine-Tuning (SFT) + LoRA
Framework	Unsloth
LoRA Rank (r)	16
LoRA Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Learning Rate	2e-4
Epochs	1
Precision	16-bit (Merged)
Hardware	Tesla T4 — Kaggle Free Tier

Dataset

Trained on the English subset of FreedomIntelligence/medical-o1-reasoning-SFT — a curated collection of complex, clinically grounded question-answer pairs structured around multi-step reasoning.

Dataset Field	Chat Role
Question	user
Complex_CoT	assistant

All samples were formatted in strict ChatML format as required by the Qwen model family.

---

Quickstart

Load the Model

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Rumiii/LWQwenMed_Human_Cognition"
)

# Enable inference mode
FastLanguageModel.for_inference(model)

Run Inference

messages = [
    {
        "role": "system",
        "content": (
            "You are a clinical reasoning assistant. "
            "Approach every medical question with careful, step-by-step analysis. "
            "Consider symptoms, relevant pathophysiology, and differential diagnoses "
            "before arriving at a conclusion."
        )
    },
    {
        "role": "user",
        "content": "Your medical question here."
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
).to("cuda")

outputs = model.generate(
    input_ids=inputs,
    max_new_tokens=512,
    temperature=0.7,
    do_sample=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Sample Reasoning Output

Click to expand — Clinical Scenario: Paradoxical Embolism

Input

Given the symptoms of sudden weakness in the left arm and leg, recent 
long-distance travel, and the presence of a swollen and tender right lower 
leg, what specific cardiac abnormality is most likely to be found upon 
further evaluation?

Model Output

Let's work through this systematically.

We are presented with sudden, unilateral left-sided weakness affecting both 
the arm and leg. This is a classic focal neurological deficit — the pattern 
suggests involvement of the right cerebral hemisphere or corticospinal tract, 
most likely embolic in origin given the acute onset.

The history of long-distance travel is a critical contextual clue. Prolonged 
immobility is a well-established risk factor for deep vein thrombosis (DVT). 
The swollen, tender right lower leg is highly consistent with this diagnosis.

Now we must connect the peripheral venous pathology to the neurological event.

Under normal circumstances, a venous thrombus travels through the right heart 
and is filtered by the pulmonary vasculature, causing a pulmonary embolism — 
not a stroke. However, if a right-to-left intracardiac shunt is present, 
a thrombus can bypass the pulmonary circulation entirely and enter the systemic 
arterial circulation. This is termed paradoxical embolism.

The most common structural substrate for paradoxical embolism is a 
Patent Foramen Ovale (PFO) — a remnant of fetal circulation present in 
approximately 25-30% of the general population. Under conditions of elevated 
right atrial pressure (as occurs with a large DVT or Valsalva maneuver), 
the PFO can transiently open, allowing a thrombus to cross into the 
left heart and embolize to the cerebral circulation.

Conclusion: The most likely cardiac finding on echocardiography in this 
patient is a Patent Foramen Ovale (PFO), serving as the anatomical conduit 
for paradoxical embolism from a DVT, resulting in the observed ischemic stroke.

---

Intended Use

This model is designed strictly for research and academic exploration in the domain of clinical natural language processing. Specific use cases include:

• Studying chain-of-thought generation in medical contexts
• Benchmarking small-scale LLMs on clinical reasoning tasks
• Exploring fine-tuning techniques for domain adaptation in healthcare NLP

---

Limitations & Ethical Considerations

This model is not a medical device and must never be used for clinical decision-making, patient diagnosis, or treatment guidance.

Limitation	Detail
Model Scale	At 0.5B parameters, the model may hallucinate or produce incomplete reasoning in rare or complex clinical scenarios
Training Duration	Trained for a single epoch — may underperform on out-of-distribution medical queries
Data Bias	Inherits biases from both the Qwen2.5 base model and the training dataset
No Clinical Validation	Outputs have not been reviewed or validated by medical professionals

---

Citation

If this model contributes to your research, please cite the base model, dataset, and framework:

@misc{lwqwenmed2026,
  author       = {Rumi Iqbal Sufi},
  title        = {LWQwenMed: Medical Chain-of-Thought Reasoning via SFT on Qwen2.5-0.5B},
  year         = {2026},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/Rumiii/LWQwenMed_Human_Cognition}
}

@misc{qwen2025,
  title        = {Qwen2.5: A Party of Foundation Models},
  author       = {Alibaba Cloud},
  year         = {2025},
  url          = {https://huggingface.co/Qwen}
}

---

Developed as part of an AI engineering research initiative.
All model outputs are experimental and intended for academic purposes only.