Instructions to use jphme/em_german_7b_v01 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jphme/em_german_7b_v01 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jphme/em_german_7b_v01")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("jphme/em_german_7b_v01")
model = AutoModelForCausalLM.from_pretrained("jphme/em_german_7b_v01")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use jphme/em_german_7b_v01 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jphme/em_german_7b_v01"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jphme/em_german_7b_v01",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/jphme/em_german_7b_v01

SGLang

How to use jphme/em_german_7b_v01 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jphme/em_german_7b_v01" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jphme/em_german_7b_v01",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jphme/em_german_7b_v01" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jphme/em_german_7b_v01",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use jphme/em_german_7b_v01 with Docker Model Runner:
```
docker model run hf.co/jphme/em_german_7b_v01
```

em_german_7b_v01

Commit History

add chat template

ae9cff0

jphme commited on Oct 27, 2023

update readme

a3f7893

jphme commited on Oct 12, 2023

update readme

13a0031

jphme commited on Oct 9, 2023

update readme

439e3e1

jphme commited on Oct 4, 2023

update readme

787e9f2

jphme commited on Oct 4, 2023

Update README.md

9f8e269

jphme commited on Sep 28, 2023

Update README.md

9babbbe

jphme commited on Sep 28, 2023

Update README.md

7246423

jphme commited on Sep 27, 2023

Create README.md

ca611ff

jphme commited on Sep 27, 2023

Add Llama2 Lincense

036c7f5

jphme commited on Sep 27, 2023

merged model upload

512d208

jphme commited on Sep 27, 2023

remove old model versions

48d00b3

Jan Philipp Harries commited on Sep 26, 2023

Add Llama2 Lincense

11ad686

jphme commited on Sep 26, 2023

merged model upload

bd80004

jphme commited on Sep 26, 2023

initial commit

90f43ba

jphme commited on Sep 26, 2023

initial commit

b7af800

jphme commited on Sep 25, 2023

Commit History

add chat template ae9cff0

update readme a3f7893

update readme 13a0031

update readme 439e3e1

update readme 787e9f2

Update README.md 9f8e269

Update README.md 9babbbe

Update README.md 7246423

Create README.md ca611ff

Add Llama2 Lincense 036c7f5

merged model upload 512d208

remove old model versions 48d00b3

Add Llama2 Lincense 11ad686

merged model upload bd80004

initial commit 90f43ba

initial commit b7af800

add chat template

ae9cff0

update readme

a3f7893

update readme

13a0031

update readme

439e3e1

update readme

787e9f2

Update README.md

9f8e269

Update README.md

9babbbe

Update README.md

7246423

Create README.md

ca611ff

Add Llama2 Lincense

036c7f5

merged model upload

512d208

remove old model versions

48d00b3

Add Llama2 Lincense

11ad686

merged model upload

bd80004

initial commit

90f43ba

initial commit

b7af800