Instructions to use nlpie/Llama2-MedTuned-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nlpie/Llama2-MedTuned-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nlpie/Llama2-MedTuned-7b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nlpie/Llama2-MedTuned-7b")
model = AutoModelForCausalLM.from_pretrained("nlpie/Llama2-MedTuned-7b")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use nlpie/Llama2-MedTuned-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nlpie/Llama2-MedTuned-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nlpie/Llama2-MedTuned-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/nlpie/Llama2-MedTuned-7b

SGLang

How to use nlpie/Llama2-MedTuned-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nlpie/Llama2-MedTuned-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nlpie/Llama2-MedTuned-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nlpie/Llama2-MedTuned-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nlpie/Llama2-MedTuned-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use nlpie/Llama2-MedTuned-7b with Docker Model Runner:
```
docker model run hf.co/nlpie/Llama2-MedTuned-7b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Model Description

Llama2-MedTuned-7b is an instruction-tuned version of the Llama2 7B model, specifically adapted for biomedical language processing tasks. It has been fine-tuned on a dataset consisting of approximately 200,000 instruction-focused samples, covering a range of biomedical and clinical NLP tasks such as Named Entity Recognition (NER), Relation Extraction (RE), and Medical Natural Language Inference (NLI).

Instruction Tuning Procedure

This model underwent instruction tuning, a process where the model is fine-tuned with detailed instructions to enhance its ability to interpret and execute specific tasks in the biomedical domain. The tuning involved the use of a comprehensive instruction-based dataset, tailor-made to align with the requirements of biomedical NLP tasks.

Model Capabilities

Llama2-MedTuned-7b demonstrates an enhanced understanding of biomedical contexts, effectively handling NER, RE, and NLI tasks. It showcases improved accuracy in generating structured outputs suitable for evaluation using conventional metrics.

Architecture

The architecture of Llama2-MedTuned-7b is based on the autoregressive transformer model Llama2 7B. This model maintains the original transformer layers and attention mechanisms, specifically adjusted to cater to the linguistic intricacies of the biomedical field.

Citation

If you utilise Llama2-MedTuned-7b in your research or application, please consider citing our paper:

@article{rohanian2024exploring,
  title = {Exploring the Effectiveness of Instruction Tuning in Biomedical Language Processing},
  author = {Rohanian, Omid and Nouriborji, Mohammadmahdi and Kouchaki, Samaneh and Nooralahzadeh, Farhad and Clifton, Lei and Clifton, David A},
  journal = {Artificial Intelligence in Medicine},
  volume = {158},
  pages = {103007},
  year = {2024},
  publisher = {Elsevier},
  doi = {10.1016/j.artmed.2024.103007},
  url = {https://www.sciencedirect.com/science/article/pii/S0933365724002495},
  issn = {0933-3657}
}

Support

If this model helps your work, you can keep the project running with a one-off or monthly contribution:
https://github.com/sponsors/nlpie-research

Downloads last month: 23

Model tree for nlpie/Llama2-MedTuned-7b

Quantizations

3 models