Instructions to use Babblu2821/gpt2-medqa-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Babblu2821/gpt2-medqa-lora with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("gpt2")
model = PeftModel.from_pretrained(base_model, "Babblu2821/gpt2-medqa-lora")

Transformers

How to use Babblu2821/gpt2-medqa-lora with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Babblu2821/gpt2-medqa-lora")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Babblu2821/gpt2-medqa-lora", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Babblu2821/gpt2-medqa-lora with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Babblu2821/gpt2-medqa-lora"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Babblu2821/gpt2-medqa-lora",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Babblu2821/gpt2-medqa-lora

SGLang

How to use Babblu2821/gpt2-medqa-lora with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Babblu2821/gpt2-medqa-lora" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Babblu2821/gpt2-medqa-lora",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Babblu2821/gpt2-medqa-lora" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Babblu2821/gpt2-medqa-lora",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Babblu2821/gpt2-medqa-lora with Docker Model Runner:
```
docker model run hf.co/Babblu2821/gpt2-medqa-lora
```

gpt2-medqa-lora / README.md

Babblu2821

docs: replace LLM-judge factual ratings with the human pass

1d5a973 verified 4 days ago

preview code

Raw

History Blame Contribute Delete

5.79 kB

metadata

base_model: gpt2
library_name: peft
pipeline_tag: text-generation
license: mit
language:
  - en
datasets:
  - keivalya/MedQuad-MedicalQnADataset
tags:
  - base_model:adapter:gpt2
  - lora
  - peft
  - transformers
  - medical
  - research-artifact

gpt2-medqa-lora

A LoRA adapter for GPT-2 (124M), fine-tuned for one epoch on MedQuAD medical Q&A. It is the baseline arm of a controlled comparison of LoRA against QLoRA — the other arm is Babblu2821/tinyllama-medqa-qlora.

Code, method and full results: https://github.com/fayazhussain2821/llm-finetuning-medqa

⚠️ Do not use this for medical information

This is a methodology demonstration, not a medical tool. Its factual reliability has been measured, and it is poor: on a blinded review of 20 held-out questions, this adapter contradicted the reference answer or invented an entity in 60% of them, scoring 1.80 out of 5 for factual soundness.

It produces fluent, confident, well-formed text that is usually wrong. Observed failures include attributing Marfan syndrome to "an infection" and inventing non-existent genes and citations. Fluency is exactly what makes this dangerous.

Do not use it for diagnosis, treatment, triage, patient-facing text, or to answer any real health question.

What it is for

Reproducing and studying a parameter-efficient fine-tuning comparison. The adapter is useful as an object of measurement — it is the smaller, cheaper arm that the project's controls are measured against. It is not useful as a question-answering model.

Training


Base model	`gpt2` (124M)
Method	LoRA (r=16, α=32, dropout=0.05)
Target modules	`c_attn`, `c_proj`
Data	MedQuAD, 16,407 pairs, 90/10 split, seed 42 → 14,766 train
Epochs	1
Learning rate	2e-4, cosine schedule, warmup ratio 0.03
Effective batch	16 (8 × 2 accumulation) — matched to the QLoRA arm
Max length	1024 tokens
Prompt format	`### Instruction:\n{question}\n\n### Response:\n`
Hardware	Colab T4

Provenance. These weights were trained on 2026-08-03 with the project's original notebook pipeline, before the code was restructured into a package. The current repository trains both arms under transformers.Trainer (the notebooks used TRL's SFTTrainer for the TinyLlama arm), so re-running the current code will not reproduce these exact weights. Every published number below was measured on these files.

Evaluation

Scored on 1,641 held-out rows, identical rows for every arm, answer span only — the prompt template differs between arms and scoring it would let boilerplate move the metric.

Bits per byte is the headline metric, not perplexity. Perplexity is per token, and GPT-2's tokenizer differs from TinyLlama's, so the two perplexities are not on one scale. Bits per byte normalises by UTF-8 bytes of the same reference text.

run	bits/byte ↓	perplexity
`gpt2` (untrained control)	0.8049	11.51
`gpt2-medqa-lora` (this model)	0.5970	6.12
`tinyllama` (untrained control)	0.6120	5.39
`tinyllama-medqa-qlora`	0.3954	2.97

Fine-tuning cut bits per byte by 25.8% against its own base model.

Generated-answer quality

Greedy decoding, ≤200 new tokens, 200 held-out questions:

	ROUGE-L F1 ↑	token F1 ↑	repeated 4-grams ↓
`gpt2` (control)	0.0797	0.1666	0.0000
this model	0.0971	0.2060	0.0005

No degeneration — the model does not loop. Its problem is that it is wrong.

Factual soundness (blinded, 1–5)

	mean ↑	contradicts reference ↓
`gpt2` (control)	1.60	70%
this model	1.80	60%

Fine-tuning produced no detectable improvement in factual accuracy (paired 95% CI −0.40 to +0.80, spanning zero) — while the automatic metrics above reported a 25.8% gain. LoRA taught this model MedQuAD's register, and register is what those metrics score.

These ratings are a human pass, rated blind to which model produced each answer, by the repository's author — one non-expert rater, not a clinician and not adjudicated by a second. An earlier LLM-judge pass over the same sheet rated this adapter far harsher (1.20 mean, 95% contradiction) but reached every identical verdict; both are published in the source repository.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("gpt2")
model = PeftModel.from_pretrained(base, "Babblu2821/gpt2-medqa-lora")
tokenizer = AutoTokenizer.from_pretrained("Babblu2821/gpt2-medqa-lora")

prompt = "### Instruction:\nWhat is anemia?\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=160, repetition_penalty=1.15)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

The prompt template matters: this adapter was trained on ### Instruction: / ### Response: and will behave worse without it.

Limitations

Not factually reliable. See the measured numbers above.
One epoch, one seed, one run — no variance estimate across training runs.
MedQuAD is NIH-sourced, US-centric, and frozen at collection time.
~5% of examples exceed GPT-2's 1024-token context and were truncated.
Evaluation compares against a single reference answer; a correct answer phrased differently scores as a miss.

License

MIT for the adapter weights. The base model and dataset carry their own licenses.