fix: correct training config (batch 2x8=16, LR 2e-5, epochs 5)

README.md

@@ -1,11 +1,31 @@
 ---
+---
 base_model: google/medgemma-4b-it
 library_name: peft
 pipeline_tag: text-generation
 license: mit
 language:
-- en
+  - en
 tags:
+  - lora
+  - transformers
+  - medical
+  - clinical-documentation
+  - soap-notes
+  - medgemma
+  - hai-def
+  - medgemma-impact-challenge
+---
+
+---
+base_model: google/medgemma-4b-it
+library_name: peft
+pipeline_tag: text-generation
+license: mit
+language:
+
+- en
+  tags:
 - lora
 - transformers
 - medical
@@ -14,8 +34,9 @@ tags:
 - medgemma
 - hai-def
 - medgemma-impact-challenge
-datasets:
-- Tushar9802/medscribe-soap-712
+
+---
+
 ---
 
 # MedScribe SOAP LoRA — Concise Clinical Note Generation
@@ -34,25 +55,27 @@ default to.
 
 **Example:**
 
-| Input transcript | "54-year-old female presenting with shortness of breath. CT chest shows filling defects in segmental branches of right lower lobe..." |
-|-----------------|---|
-| **Base MedGemma** | ~200 words, textbook prose, over-specified plan with 6-8 items |
-| **This adapter** | ~104 words, clinical shorthand ("54 yo F c/o SOB"), focused 2-4 item plan |
+| Input transcript        | "54-year-old female presenting with shortness of breath. CT chest shows filling defects in segmental branches of right lower lobe..." |
+| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
+| **Base MedGemma** | ~200 words, textbook prose, over-specified plan with 6-8 items                                                                        |
+| **This adapter**  | ~104 words, clinical shorthand ("54 yo F c/o SOB"), focused 2-4 item plan                                                             |
 
 ## Key Metrics
 
-| Metric | Base MedGemma | With This Adapter |
-|--------|--------------|-------------------|
-| Avg word count | ~200+ | 104 |
-| Section completeness (S/O/A/P) | 85-95% | 100% |
-| Hallucinated findings | 5-10% | 0% |
-| WNL shortcuts | Present | 0% |
-| Clinical style | Textbook verbose | Shorthand |
-| PLAN items | 4-8 | 2-4 (focused) |
-| Quality score | — | 90/100 |
+| Metric                         | Base MedGemma    | With This Adapter |
+| ------------------------------ | ---------------- | ----------------- |
+| Avg word count                 | ~200+            | 104               |
+| Section completeness (S/O/A/P) | 85-95%           | 100%              |
+| Hallucinated findings          | 5-10%            | 0%                |
+| WNL shortcuts                  | Present          | 0%                |
+| Clinical style                 | Textbook verbose | Shorthand         |
+| PLAN items                     | 4-8              | 2-4 (focused)     |
+| Quality score                  | —               | 90/100            |
 
 ## Usage
 
+python
+
 ```python
 from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
 from peft import PeftModel
@@ -67,18 +90,18 @@ bnb_config = BitsAndBytesConfig(
 )
 
 base_model = AutoModelForCausalLM.from_pretrained(
-    "google/medgemma-4b-it",
+"google/medgemma-4b-it",
     quantization_config=bnb_config,
     device_map="auto",
 )
 tokenizer = AutoTokenizer.from_pretrained("google/medgemma-4b-it")
 
 # Load LoRA adapter
-model = PeftModel.from_pretrained(base_model, "Tushar-9802/medscribe-soap-lora")
+model = PeftModel.from_pretrained(base_model,"Tushar-9802/medscribe-soap-lora")
 model.eval()
 
 # Generate SOAP note
-prompt = """You are a clinical documentation assistant. Convert the following medical
+prompt ="""You are a clinical documentation assistant. Convert the following medical
 text into a structured SOAP note.
 
 MEDICAL TEXT:
@@ -96,12 +119,12 @@ SOAP NOTE:"""
 inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
 with torch.inference_mode():
     outputs = model.generate(
-        **inputs,
+**inputs,
         max_new_tokens=400,
         min_new_tokens=150,
         do_sample=False,
         use_cache=True,
-    )
+)
 result = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
 print(result)
 ```
@@ -114,36 +137,37 @@ print(result)
 Dataset: [Tushar-9802/medscribe-soap-712](https://huggingface.co/datasets/Tushar-9802/medscribe-soap-712)
 
 Each sample enforces:
-- "Not documented in source" for any finding absent from the input transcript
-- Zero WNL (Within Normal Limits) shortcuts — every finding explicitly stated
-- Concise clinical shorthand style
-- PLAN with specific, actionable items
+
+* "Not documented in source" for any finding absent from the input transcript
+* Zero WNL (Within Normal Limits) shortcuts — every finding explicitly stated
+* Concise clinical shorthand style
+* PLAN with specific, actionable items
 
 ### Training Configuration
 
-| Parameter | Value |
-|-----------|-------|
-| Base model | google/medgemma-4b-it |
-| Method | LoRA |
-| Rank | 16 |
-| Alpha | 32 |
-| Dropout | 0.05 |
-| Target modules | All attention layers |
-| Trainable parameters | ~4.2M (0.1% of 4B base) |
-| Batch size | 8 (× 4 gradient accumulation = effective 32) |
-| Learning rate | 2e-4 |
-| Epochs | 3 |
-| Precision | BFloat16 |
-| Quantization | 4-bit NF4 during training |
-| Hardware | NVIDIA RTX 5070 Ti (16GB VRAM) |
+| Parameter            | Value                                         |
+| -------------------- | --------------------------------------------- |
+| Base model           | google/medgemma-4b-it                         |
+| Method               | LoRA                                          |
+| Rank                 | 16                                            |
+| Alpha                | 32                                            |
+| Dropout              | 0.1                                           |
+| Target modules       | All attention layers                          |
+| Trainable parameters | ~4.2M (0.1% of 4B base)                       |
+| Batch size           | 2 (× 8 gradient accumulation = effective 16)  |
+| Learning rate        | 2e-5                                          |
+| Epochs               | 5 (early stopping patience: 2)                |
+| Precision            | BFloat16                                      |
+| Quantization         | 4-bit NF4 during training                     |
+| Hardware             | NVIDIA RTX 5070 Ti (16GB VRAM)                |
 
 ### Training Results
 
-| Metric | Value |
-|--------|-------|
-| Training loss | 0.828 |
-| Validation loss | 0.782 |
-| Overfitting | None (val < train) |
+| Metric          | Value              |
+| --------------- | ------------------ |
+| Training loss   | 0.828              |
+| Validation loss | 0.782              |
+| Overfitting     | None (val < train) |
 
 ## Anti-Hallucination Behavior
 
@@ -155,19 +179,19 @@ missing is far safer than a plausible-sounding fabrication.
 
 ## Intended Use
 
-- Converting medical encounter transcripts to structured SOAP notes
-- Clinical documentation assistance (with physician review)
-- Research and demonstration of efficient medical LLM fine-tuning
+* Converting medical encounter transcripts to structured SOAP notes
+* Clinical documentation assistance (with physician review)
+* Research and demonstration of efficient medical LLM fine-tuning
 
 ## Limitations
 
-- **English only**
-- **Research prototype** — not validated for clinical use in any jurisdiction
-- **Synthetic training data** — 712 samples generated by GPT-4o Mini, not
+* **English only**
+* **Research prototype** — not validated for clinical use in any jurisdiction
+* **Synthetic training data** — 712 samples generated by GPT-4o Mini, not
   from real clinical encounters
-- **Requires physician review** — all generated notes must be reviewed and
+* **Requires physician review** — all generated notes must be reviewed and
   approved by a licensed clinician before use in patient care
-- **Inference speed** — ~25 seconds per note on RTX 5070 Ti with 4-bit
+* **Inference speed** — ~25 seconds per note on RTX 5070 Ti with 4-bit
   quantization
 
 ## Part Of
@@ -179,23 +203,25 @@ this fine-tuned MedGemma adapter (SOAP generation), and base MedGemma
 
 ## Framework Versions
 
-- PEFT 0.18.1
-- Transformers 4.52+
-- PyTorch 2.8+ (nightly for Blackwell/SM 12.0)
-- bitsandbytes 0.45+
+* PEFT 0.18.1
+* Transformers 4.52+
+* PyTorch 2.8+ (nightly for Blackwell/SM 12.0)
+* bitsandbytes 0.45+
 
 ## Citation
 
+bibtex
+
 ```bibtex
 @misc{medscribe2026,
   author = {Tushar},
   title = {MedScribe: Concise Clinical Documentation via Fine-tuned MedGemma},
   year = {2026},
   publisher = {HuggingFace},
-  url = {https://huggingface.co/Tushar9802/medscribe-soap-lora}
+  url = {https://huggingface.co/Tushar-9802/medscribe-soap-lora}
 }
 ```
 
 ## Contact
 
-GitHub: [@Tushar-9802](https://github.com/Tushar-9802)
+GitHub: [@Tushar-9802](https://github.com/Tushar-9802)