Upload README.md with huggingface_hub

README.md

@@ -7,6 +7,7 @@ tags:
   - lora
   - peft
   - medgemma
+  - gguf
 language:
   - en
 library_name: peft
@@ -24,20 +25,45 @@ Trained models for clinical note simplification - translating medical documents
 | **gemma-2b-dpo** | gemma-2-2b-it | DPO comparison | **73%** | **82%** | 61% |
 | **gemma-9b-dpo** | gemma-2-9b-it | Teacher model | 79% | 91% | 70% |
 
-## Usage
+## GGUF for Mobile/Local Inference
+
+Pre-quantized GGUF models (Q4_K_M, ~1.6GB each) for llama.cpp, Ollama, LM Studio:
+
+| File | Description | Download |
+|------|-------------|----------|
+| `gguf/gemma-2b-distilled-q4_k_m.gguf` | Distilled model (better patient communication) | [Download](https://huggingface.co/dejori/note-explain/resolve/main/gguf/gemma-2b-distilled-q4_k_m.gguf) |
+| `gguf/gemma-2b-dpo-q4_k_m.gguf` | DPO model (higher accuracy) | [Download](https://huggingface.co/dejori/note-explain/resolve/main/gguf/gemma-2b-dpo-q4_k_m.gguf) |
+
+### Quick Start with Ollama
+
+```bash
+# Download and run
+ollama run hf.co/dejori/note-explain:gemma-2b-distilled-q4_k_m.gguf
+```
+
+### Quick Start with llama.cpp
+
+```bash
+# Download
+wget https://huggingface.co/dejori/note-explain/resolve/main/gguf/gemma-2b-distilled-q4_k_m.gguf
+
+# Run
+./llama-cli -m gemma-2b-distilled-q4_k_m.gguf -p "Simplify this clinical note for a patient: [your note]"
+```
+
+## LoRA Adapters
+
+For fine-tuning or full-precision inference:
 
 ```python
 from peft import PeftModel
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
-# Load the distilled model (recommended for deployment)
+# Load the distilled model
 base_model = AutoModelForCausalLM.from_pretrained("google/gemma-2-2b-it")
 tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-2b-it")
 model = PeftModel.from_pretrained(base_model, "dejori/note-explain", subfolder="gemma-2b-distilled")
 
-# Or load the DPO model (higher accuracy)
-model = PeftModel.from_pretrained(base_model, "dejori/note-explain", subfolder="gemma-2b-dpo")
-
 # Generate
 prompt = "Simplify this clinical note for a patient:\n\n[clinical note]\n\nSimplified version:"
 inputs = tokenizer(prompt, return_tensors="pt")
@@ -54,17 +80,6 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 
 Training data: [dejori/note-explain](https://huggingface.co/datasets/dejori/note-explain)
 
-## Citation
-
-```bibtex
-@misc{noteexplain2026,
-  title={NoteExplain: Privacy-First Clinical Note Simplification},
-  author={Dejori, Mathaeus},
-  year={2026},
-  publisher={HuggingFace}
-}
-```
-
 ## License
 
 Apache 2.0