NorthernTribe-Research commited on
Commit
ca2e1db
·
verified ·
1 Parent(s): 2ec8c15

Add README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -0
README.md CHANGED
@@ -47,6 +47,9 @@ This model is designed for:
47
  - **Base Model**: [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
48
  - **Dataset**: `NorthernTribe-Research/comprehensive-healthbench-v2` (1.4M examples)
49
  - **Method**: QLoRA Fine-tuning via [Unsloth](https://github.com/unslothai/unsloth)
 
 
 
50
  - **Infrastructure**: Trained on Nvidia A100/H100 GPUs.
51
 
52
  ## Usage
 
47
  - **Base Model**: [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)
48
  - **Dataset**: `NorthernTribe-Research/comprehensive-healthbench-v2` (1.4M examples)
49
  - **Method**: QLoRA Fine-tuning via [Unsloth](https://github.com/unslothai/unsloth)
50
+ - **SOTA Techniques**:
51
+ - **NEFTune (Noisy Embeddings Fine Tuning)**: Implemented to improve model generalization and robustness.
52
+ - **Teacher-Student Distillation**: System prompts strictly enforce the model to act as an "Expert Medical Teacher", engaging in deep Chain-of-Thought reasoning for every response.
53
  - **Infrastructure**: Trained on Nvidia A100/H100 GPUs.
54
 
55
  ## Usage