Instructions to use Pennlaine/Mistral-7B-v02-Entity-Extraction with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Pennlaine/Mistral-7B-v02-Entity-Extraction with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Pennlaine/Mistral-7B-v02-Entity-Extraction")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Pennlaine/Mistral-7B-v02-Entity-Extraction", dtype="auto")

PEFT
How to use Pennlaine/Mistral-7B-v02-Entity-Extraction with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Pennlaine/Mistral-7B-v02-Entity-Extraction with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Pennlaine/Mistral-7B-v02-Entity-Extraction"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Pennlaine/Mistral-7B-v02-Entity-Extraction",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Pennlaine/Mistral-7B-v02-Entity-Extraction

SGLang

How to use Pennlaine/Mistral-7B-v02-Entity-Extraction with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Pennlaine/Mistral-7B-v02-Entity-Extraction" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Pennlaine/Mistral-7B-v02-Entity-Extraction",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Pennlaine/Mistral-7B-v02-Entity-Extraction" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Pennlaine/Mistral-7B-v02-Entity-Extraction",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Pennlaine/Mistral-7B-v02-Entity-Extraction with Docker Model Runner:
```
docker model run hf.co/Pennlaine/Mistral-7B-v02-Entity-Extraction
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Model Introduction

This model is based on mistral-community/Mistral-7B-v0.2 and has been LoRA finetuned on an English language instruction dataset.

It aims to help extract keywords/entities from unstructured data into Json formatted data.

Traning Details

GPU: NVIDIA_L4 x 4, 100 GB Time: 5 mins Platform: Google Cloud Vertex AI

Prompt Format

<s>[INST]{question}Answer the question, extract the {entities}, and return in Json format.[/INST]```json

Since this corresponds to the training data, the model performs best when the prompt is in this format. You could also simply input your question and most of the time it works too!🥰

E.g:

Input

prompt = '''<s>[INST]Type 2 Diabetes Mellitus is a chronic metabolic disorder characterized by insulin resistance and relative insulin deficiency. This condition leads to chronic hyperglycemia, which can cause significant damage to various body systems over time. What is the abbreviation? Answer the question, extract the disorder, type of disorder, causes, effect and return in Json format.[/INST]```json'''

Output

[INST]Type 2 Diabetes Mellitus is a chronic metabolic disorder characterized by insulin resistance and relative insulin deficiency. This condition leads to chronic hyperglycemia, which can cause significant damage to various body systems over time. Write a short summary of how to treat patients with diabetes. Answer the question, extract the disorder, type of disorder, causes, effect, ICD Code, and return in Json format.[/INST]```json
{
    "question": "Type 2 Diabetes Mellitus is a chronic metabolic disorder characterized by insulin resistance and relative insulin deficiency. This condition leads to chronic hyperglycemia, which can cause significant damage to various body systems over time. Write a short summary of how to treat patients with diabetes.",
    "answer": "Treatment for diabetes typically involves a combination of lifestyle modifications, medication, and insulin therapy.",
    "entities": [
        {
            "Disorder": "Type 2 Diabetes Mellitus"
        },
        {
            "Type of Disorder": "Chronic Metabolic Disorder"
        },
        {
            "Causes": "Insulin Resistance and Relative Insulin Deficiency"
        },
        {
            "Effect": "Chronic Hyperglycemia"
        },
        {
            "ICD Code": "E11"
        }
    ]
}
```

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "Pennlaine/Mistral-7B-v02-Entity-Extraction"

tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path)

prompt = '''<s>[INST]John Doe, a 45-year-old male, presents with a two-week history of progressive chest pain and shortness of breath. The chest pain is described as a tight, squeezing sensation located centrally and radiating to the left arm and jaw. It is aggravated by physical exertion and alleviated by rest. The patient reports associated symptoms of palpitations, diaphoresis, and nausea. He has a history of hypertension, hyperlipidemia, and type 2 diabetes mellitus. His family history is significant for myocardial infarction in his father at age 60. The patient has a 20-pack-year smoking history but quit 5 years ago. He occasionally consumes alcohol and leads a sedentary lifestyle with a diet high in processed foods and red meat. Current medications include metformin, lisinopril, atorvastatin, and aspirin.
Extract the name, age, symptoms, medical history, family history, father death age, medication history  and return in Json format.[/INST]```json'''

inputs = tokenizer.encode(prompt, return_tensors='pt')
outputs = model.generate(inputs, max_new_tokens=max_new_tokens, num_return_sequences=1)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(response)

Downloads last month: -; Downloads are not tracked for this model. How to track