Instructions to use mao1207/BioMed-VITAL-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mao1207/BioMed-VITAL-models with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mao1207/BioMed-VITAL-models")

# Load model directly
from transformers import AutoProcessor, AutoModelForCausalLM

processor = AutoProcessor.from_pretrained("mao1207/BioMed-VITAL-models")
model = AutoModelForCausalLM.from_pretrained("mao1207/BioMed-VITAL-models")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mao1207/BioMed-VITAL-models with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mao1207/BioMed-VITAL-models"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mao1207/BioMed-VITAL-models",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/mao1207/BioMed-VITAL-models

SGLang

How to use mao1207/BioMed-VITAL-models with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mao1207/BioMed-VITAL-models" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mao1207/BioMed-VITAL-models",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mao1207/BioMed-VITAL-models" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mao1207/BioMed-VITAL-models",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use mao1207/BioMed-VITAL-models with Docker Model Runner:
```
docker model run hf.co/mao1207/BioMed-VITAL-models
```

Checkpoint queries.

by jusepv - opened Jan 16, 2025

Discussion

jusepv

Jan 16, 2025

I was wondering if the weights in the model you uploaded are the weights you trained?

When I download and load the weights, some layers seem to be untrained, are they vanilla LLaVA weights?

I've left multiple inquiries, please confirm.

mao1207

Owner Jan 17, 2025

•

edited Jan 17, 2025

Yes, our weights are the vanilla LLaVA weights. And our training followed a similar approach to LLaVA-Med, using delta training. We then merged the delta weights with the original LLaMA weights. We recommend referring to the specific instructions in this link, which may help resolve your issue.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment