Text Generation
PEFT
English
qwen2
medical
reasoning
qwen
healthbench
clinical-reasoning
chain-of-thought
Instructions to use NorthernTribe-Research/NorthernTribe-Medical-Reasoning-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use NorthernTribe-Research/NorthernTribe-Medical-Reasoning-7B with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Add README.md
Browse files
README.md
CHANGED
|
@@ -47,6 +47,9 @@ This model is designed for:
|
|
| 47 |
- **Base Model**: [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
|
| 48 |
- **Dataset**: `NorthernTribe-Research/comprehensive-healthbench-v2` (1.4M examples)
|
| 49 |
- **Method**: QLoRA Fine-tuning via [Unsloth](https://github.com/unslothai/unsloth)
|
|
|
|
|
|
|
|
|
|
| 50 |
- **Infrastructure**: Trained on Nvidia A100/H100 GPUs.
|
| 51 |
|
| 52 |
## Usage
|
|
|
|
| 47 |
- **Base Model**: [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
|
| 48 |
- **Dataset**: `NorthernTribe-Research/comprehensive-healthbench-v2` (1.4M examples)
|
| 49 |
- **Method**: QLoRA Fine-tuning via [Unsloth](https://github.com/unslothai/unsloth)
|
| 50 |
+
- **SOTA Techniques**:
|
| 51 |
+
- **NEFTune (Noisy Embeddings Fine Tuning)**: Implemented to improve model generalization and robustness.
|
| 52 |
+
- **Teacher-Student Distillation**: System prompts strictly enforce the model to act as an "Expert Medical Teacher", engaging in deep Chain-of-Thought reasoning for every response.
|
| 53 |
- **Infrastructure**: Trained on Nvidia A100/H100 GPUs.
|
| 54 |
|
| 55 |
## Usage
|