Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -186,12 +186,15 @@ The training dataset consisted of 119,117 carefully curated entries focused on s
|
|
| 186 |
- Real-world scenarios and practical advice situations
|
| 187 |
- Educational Q&A formatted for instruction following
|
| 188 |
|
| 189 |
-
All training data was filtered and
|
| 190 |
-
-
|
|
|
|
| 191 |
- Appropriate tone and sensitivity for SCI community
|
| 192 |
- Removal of potentially harmful or dangerous advice
|
| 193 |
- Proper medical disclaimers and context
|
| 194 |
|
|
|
|
|
|
|
| 195 |
### Training Procedure
|
| 196 |
|
| 197 |
The model was trained using a two-phase approach with QLoRA (Quantized Low-Rank Adaptation):
|
|
@@ -246,9 +249,11 @@ Training data underwent extensive preprocessing:
|
|
| 246 |
|
| 247 |
The model was evaluated using:
|
| 248 |
- Held-out test set of SCI-related questions (500 samples)
|
| 249 |
-
-
|
| 250 |
-
-
|
| 251 |
-
-
|
|
|
|
|
|
|
| 252 |
|
| 253 |
#### Factors
|
| 254 |
|
|
@@ -261,18 +266,18 @@ Evaluation considered multiple factors:
|
|
| 261 |
|
| 262 |
#### Metrics
|
| 263 |
|
| 264 |
-
- **Medical accuracy score**:
|
| 265 |
-
- **Appropriateness rating**:
|
| 266 |
- **Response relevance**: SCI-specific context understanding (82% relevance score)
|
| 267 |
-
- **Safety compliance**:
|
| 268 |
- **Response quality**: Perplexity improvements over base model for SCI domain
|
| 269 |
|
| 270 |
### Results
|
| 271 |
|
| 272 |
**Quantitative Results:**
|
| 273 |
- 40% improvement in SCI domain perplexity over base model
|
| 274 |
-
-
|
| 275 |
-
- 95% safety compliance (no harmful medical advice detected)
|
| 276 |
- 82% average relevance score for SCI-specific contexts
|
| 277 |
|
| 278 |
**Qualitative Results:**
|
|
@@ -281,10 +286,10 @@ Evaluation considered multiple factors:
|
|
| 281 |
- Consistent inclusion of medical disclaimers
|
| 282 |
- Good balance between being helpful and cautious about medical advice
|
| 283 |
|
| 284 |
-
**
|
| 285 |
-
-
|
| 286 |
-
-
|
| 287 |
-
-
|
| 288 |
|
| 289 |
## Environmental Impact
|
| 290 |
|
|
@@ -359,7 +364,8 @@ basiphobe. (2025). *SCI Assistant: A Specialized AI Assistant for Spinal Cord In
|
|
| 359 |
|
| 360 |
**Primary Author:** basiphobe
|
| 361 |
**Model Development:** Individual research project for SCI community support
|
| 362 |
-
**
|
|
|
|
| 363 |
|
| 364 |
## Model Card Contact
|
| 365 |
|
|
|
|
| 186 |
- Real-world scenarios and practical advice situations
|
| 187 |
- Educational Q&A formatted for instruction following
|
| 188 |
|
| 189 |
+
All training data was filtered and curated to ensure:
|
| 190 |
+
- Sources from reputable medical organizations and healthcare professionals
|
| 191 |
+
- Content originally created or reviewed by medical professionals in the SCI field
|
| 192 |
- Appropriate tone and sensitivity for SCI community
|
| 193 |
- Removal of potentially harmful or dangerous advice
|
| 194 |
- Proper medical disclaimers and context
|
| 195 |
|
| 196 |
+
**Note**: While the source materials were created by medical professionals, this model itself has not undergone independent medical validation.
|
| 197 |
+
|
| 198 |
### Training Procedure
|
| 199 |
|
| 200 |
The model was trained using a two-phase approach with QLoRA (Quantized Low-Rank Adaptation):
|
|
|
|
| 249 |
|
| 250 |
The model was evaluated using:
|
| 251 |
- Held-out test set of SCI-related questions (500 samples)
|
| 252 |
+
- Manual review of response quality and appropriateness
|
| 253 |
+
- Comparative analysis against general-purpose models on SCI topics
|
| 254 |
+
- Assessment of domain-specific knowledge retention
|
| 255 |
+
|
| 256 |
+
**Note**: Evaluation was conducted by the model developer, not independent medical professionals.
|
| 257 |
|
| 258 |
#### Factors
|
| 259 |
|
|
|
|
| 266 |
|
| 267 |
#### Metrics
|
| 268 |
|
| 269 |
+
- **Medical accuracy score**: Based on consistency with source medical literature (not independently validated)
|
| 270 |
+
- **Appropriateness rating**: Developer assessment of tone and sensitivity (4.2/5.0 subjective rating)
|
| 271 |
- **Response relevance**: SCI-specific context understanding (82% relevance score)
|
| 272 |
+
- **Safety compliance**: No obviously harmful medical advice detected in test samples
|
| 273 |
- **Response quality**: Perplexity improvements over base model for SCI domain
|
| 274 |
|
| 275 |
### Results
|
| 276 |
|
| 277 |
**Quantitative Results:**
|
| 278 |
- 40% improvement in SCI domain perplexity over base model
|
| 279 |
+
- Responses demonstrate consistency with source medical literature
|
| 280 |
+
- 95% safety compliance (no obviously harmful medical advice detected)
|
| 281 |
- 82% average relevance score for SCI-specific contexts
|
| 282 |
|
| 283 |
**Qualitative Results:**
|
|
|
|
| 286 |
- Consistent inclusion of medical disclaimers
|
| 287 |
- Good balance between being helpful and cautious about medical advice
|
| 288 |
|
| 289 |
+
**Limitations of Evaluation:**
|
| 290 |
+
- Evaluation conducted by model developer, not independent medical experts
|
| 291 |
+
- No formal clinical validation or testing with SCI patients
|
| 292 |
+
- Results based on consistency with training sources, not independent medical verification
|
| 293 |
|
| 294 |
## Environmental Impact
|
| 295 |
|
|
|
|
| 364 |
|
| 365 |
**Primary Author:** basiphobe
|
| 366 |
**Model Development:** Individual research project for SCI community support
|
| 367 |
+
**Data Sources:** Curated from medical literature and educational materials created by healthcare professionals
|
| 368 |
+
**Validation Status:** Model has not undergone independent medical professional validation
|
| 369 |
|
| 370 |
## Model Card Contact
|
| 371 |
|