basiphobe commited on
Commit
a9f8dec
·
verified ·
1 Parent(s): b5578db

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +21 -15
README.md CHANGED
@@ -186,12 +186,15 @@ The training dataset consisted of 119,117 carefully curated entries focused on s
186
  - Real-world scenarios and practical advice situations
187
  - Educational Q&A formatted for instruction following
188
 
189
- All training data was filtered and validated to ensure:
190
- - Medical accuracy and reliability
 
191
  - Appropriate tone and sensitivity for SCI community
192
  - Removal of potentially harmful or dangerous advice
193
  - Proper medical disclaimers and context
194
 
 
 
195
  ### Training Procedure
196
 
197
  The model was trained using a two-phase approach with QLoRA (Quantized Low-Rank Adaptation):
@@ -246,9 +249,11 @@ Training data underwent extensive preprocessing:
246
 
247
  The model was evaluated using:
248
  - Held-out test set of SCI-related questions (500 samples)
249
- - Real-world scenarios from SCI community feedback
250
- - Medical professional review of sample responses
251
- - Comparative analysis against general-purpose models
 
 
252
 
253
  #### Factors
254
 
@@ -261,18 +266,18 @@ Evaluation considered multiple factors:
261
 
262
  #### Metrics
263
 
264
- - **Medical accuracy score**: Professional healthcare review (85% accuracy on factual medical content)
265
- - **Appropriateness rating**: Community feedback (4.2/5.0 average rating)
266
  - **Response relevance**: SCI-specific context understanding (82% relevance score)
267
- - **Safety compliance**: Zero harmful medical advice detected in test samples
268
  - **Response quality**: Perplexity improvements over base model for SCI domain
269
 
270
  ### Results
271
 
272
  **Quantitative Results:**
273
  - 40% improvement in SCI domain perplexity over base model
274
- - 85% medical accuracy on factual SCI content (healthcare professional review)
275
- - 95% safety compliance (no harmful medical advice detected)
276
  - 82% average relevance score for SCI-specific contexts
277
 
278
  **Qualitative Results:**
@@ -281,10 +286,10 @@ Evaluation considered multiple factors:
281
  - Consistent inclusion of medical disclaimers
282
  - Good balance between being helpful and cautious about medical advice
283
 
284
- **Community Feedback:**
285
- - SCI community members rated responses as more relevant and helpful compared to general models
286
- - Healthcare professionals noted improved medical terminology usage
287
- - Consistent appropriate referrals to medical professionals when needed
288
 
289
  ## Environmental Impact
290
 
@@ -359,7 +364,8 @@ basiphobe. (2025). *SCI Assistant: A Specialized AI Assistant for Spinal Cord In
359
 
360
  **Primary Author:** basiphobe
361
  **Model Development:** Individual research project for SCI community support
362
- **Medical Consultation:** Content reviewed by healthcare professionals familiar with SCI care
 
363
 
364
  ## Model Card Contact
365
 
 
186
  - Real-world scenarios and practical advice situations
187
  - Educational Q&A formatted for instruction following
188
 
189
+ All training data was filtered and curated to ensure:
190
+ - Sources from reputable medical organizations and healthcare professionals
191
+ - Content originally created or reviewed by medical professionals in the SCI field
192
  - Appropriate tone and sensitivity for SCI community
193
  - Removal of potentially harmful or dangerous advice
194
  - Proper medical disclaimers and context
195
 
196
+ **Note**: While the source materials were created by medical professionals, this model itself has not undergone independent medical validation.
197
+
198
  ### Training Procedure
199
 
200
  The model was trained using a two-phase approach with QLoRA (Quantized Low-Rank Adaptation):
 
249
 
250
  The model was evaluated using:
251
  - Held-out test set of SCI-related questions (500 samples)
252
+ - Manual review of response quality and appropriateness
253
+ - Comparative analysis against general-purpose models on SCI topics
254
+ - Assessment of domain-specific knowledge retention
255
+
256
+ **Note**: Evaluation was conducted by the model developer, not independent medical professionals.
257
 
258
  #### Factors
259
 
 
266
 
267
  #### Metrics
268
 
269
+ - **Medical accuracy score**: Based on consistency with source medical literature (not independently validated)
270
+ - **Appropriateness rating**: Developer assessment of tone and sensitivity (4.2/5.0 subjective rating)
271
  - **Response relevance**: SCI-specific context understanding (82% relevance score)
272
+ - **Safety compliance**: No obviously harmful medical advice detected in test samples
273
  - **Response quality**: Perplexity improvements over base model for SCI domain
274
 
275
  ### Results
276
 
277
  **Quantitative Results:**
278
  - 40% improvement in SCI domain perplexity over base model
279
+ - Responses demonstrate consistency with source medical literature
280
+ - 95% safety compliance (no obviously harmful medical advice detected)
281
  - 82% average relevance score for SCI-specific contexts
282
 
283
  **Qualitative Results:**
 
286
  - Consistent inclusion of medical disclaimers
287
  - Good balance between being helpful and cautious about medical advice
288
 
289
+ **Limitations of Evaluation:**
290
+ - Evaluation conducted by model developer, not independent medical experts
291
+ - No formal clinical validation or testing with SCI patients
292
+ - Results based on consistency with training sources, not independent medical verification
293
 
294
  ## Environmental Impact
295
 
 
364
 
365
  **Primary Author:** basiphobe
366
  **Model Development:** Individual research project for SCI community support
367
+ **Data Sources:** Curated from medical literature and educational materials created by healthcare professionals
368
+ **Validation Status:** Model has not undergone independent medical professional validation
369
 
370
  ## Model Card Contact
371