Add README.md
Browse files
README.md
CHANGED
|
@@ -47,6 +47,9 @@ This model is designed for:
|
|
| 47 |
- **Base Model**: [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
|
| 48 |
- **Dataset**: `NorthernTribe-Research/comprehensive-healthbench-v2` (1.4M examples)
|
| 49 |
- **Method**: QLoRA Fine-tuning via [Unsloth](https://github.com/unslothai/unsloth)
|
|
|
|
|
|
|
|
|
|
| 50 |
- **Infrastructure**: Trained on Nvidia A100/H100 GPUs.
|
| 51 |
|
| 52 |
## Usage
|
|
|
|
| 47 |
- **Base Model**: [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
|
| 48 |
- **Dataset**: `NorthernTribe-Research/comprehensive-healthbench-v2` (1.4M examples)
|
| 49 |
- **Method**: QLoRA Fine-tuning via [Unsloth](https://github.com/unslothai/unsloth)
|
| 50 |
+
- **SOTA Techniques**:
|
| 51 |
+
- **NEFTune (Noisy Embeddings Fine Tuning)**: Implemented to improve model generalization and robustness.
|
| 52 |
+
- **Teacher-Student Distillation**: System prompts strictly enforce the model to act as an "Expert Medical Teacher", engaging in deep Chain-of-Thought reasoning for every response.
|
| 53 |
- **Infrastructure**: Trained on Nvidia A100/H100 GPUs.
|
| 54 |
|
| 55 |
## Usage
|