Instructions to use Babblu2821/gpt2-medqa-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Babblu2821/gpt2-medqa-lora with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("gpt2")
model = PeftModel.from_pretrained(base_model, "Babblu2821/gpt2-medqa-lora")

Transformers

How to use Babblu2821/gpt2-medqa-lora with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Babblu2821/gpt2-medqa-lora")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Babblu2821/gpt2-medqa-lora", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Babblu2821/gpt2-medqa-lora with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Babblu2821/gpt2-medqa-lora"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Babblu2821/gpt2-medqa-lora",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Babblu2821/gpt2-medqa-lora

SGLang

How to use Babblu2821/gpt2-medqa-lora with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Babblu2821/gpt2-medqa-lora" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Babblu2821/gpt2-medqa-lora",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Babblu2821/gpt2-medqa-lora" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Babblu2821/gpt2-medqa-lora",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Babblu2821/gpt2-medqa-lora with Docker Model Runner:
```
docker model run hf.co/Babblu2821/gpt2-medqa-lora
```

Babblu2821 commited on 4 days ago

Commit

1d5a973

verified ·

1 Parent(s): f532a20

docs: replace LLM-judge factual ratings with the human pass

Browse files

Files changed (1) hide show

README.md +10 -7

README.md CHANGED Viewed

@@ -29,8 +29,8 @@ Code, method and full results: **https://github.com/fayazhussain2821/llm-finetun
 >
 > This is a **methodology demonstration**, not a medical tool. Its factual reliability
 > has been measured, and it is poor: on a blinded review of 20 held-out questions, this
-> adapter **contradicted the reference answer or invented an entity in 95% of them**,
-> scoring **1.20 out of 5** for factual soundness.
 >
 > It produces fluent, confident, well-formed text that is usually wrong. Observed
 > failures include attributing Marfan syndrome to "an infection" and inventing
@@ -101,16 +101,19 @@ No degeneration — the model does not loop. Its problem is that it is wrong.
 | | mean ↑ | contradicts reference ↓ |
 |---|---|---|
-| `gpt2` (control) | 1.45 | 90% |
-| **this model** | **1.20** | **95%** |
 Fine-tuning produced **no detectable improvement in factual accuracy** (paired 95% CI
-−0.65 to +0.10, spanning zero) — while the automatic metrics above reported a 25.8%
 gain. LoRA taught this model MedQuAD's *register*, and register is what those metrics
 score.
-These ratings were produced by an LLM judge, not a clinician, and are labelled as such
-in the source repository. They are a strong prior, not a clinical assessment.
 ## Usage

 >
 > This is a **methodology demonstration**, not a medical tool. Its factual reliability
 > has been measured, and it is poor: on a blinded review of 20 held-out questions, this
+> adapter **contradicted the reference answer or invented an entity in 60% of them**,
+> scoring **1.80 out of 5** for factual soundness.
 >
 > It produces fluent, confident, well-formed text that is usually wrong. Observed
 > failures include attributing Marfan syndrome to "an infection" and inventing
 | | mean ↑ | contradicts reference ↓ |
 |---|---|---|
+| `gpt2` (control) | 1.60 | 70% |
+| **this model** | **1.80** | **60%** |
 Fine-tuning produced **no detectable improvement in factual accuracy** (paired 95% CI
+−0.40 to +0.80, spanning zero) — while the automatic metrics above reported a 25.8%
 gain. LoRA taught this model MedQuAD's *register*, and register is what those metrics
 score.
+These ratings are a human pass, rated blind to which model produced each answer, by
+the repository's author — one non-expert rater, not a clinician and not adjudicated by
+a second. An earlier LLM-judge pass over the same sheet rated this adapter far harsher
+(1.20 mean, 95% contradiction) but reached every identical verdict; both are published
+in the source repository.
 ## Usage