Instructions to use Sumail/Eurus8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Sumail/Eurus8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Sumail/Eurus8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Sumail/Eurus8")
model = AutoModelForCausalLM.from_pretrained("Sumail/Eurus8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Sumail/Eurus8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Sumail/Eurus8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sumail/Eurus8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Sumail/Eurus8

SGLang

How to use Sumail/Eurus8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Sumail/Eurus8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sumail/Eurus8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Sumail/Eurus8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sumail/Eurus8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Sumail/Eurus8 with Docker Model Runner:
```
docker model run hf.co/Sumail/Eurus8
```

Sumail commited on Aug 22, 2024

Commit

579264a

verified ·

1 Parent(s): bfea208

Upload folder using huggingface_hub

Browse files

Files changed (7) hide show

README.md +7 -7
config.json +1 -1
mergekit_config.yml +3 -3
model-00001-of-00004.safetensors +1 -1
model-00002-of-00004.safetensors +1 -1
model-00003-of-00004.safetensors +1 -1
model-00004-of-00004.safetensors +1 -1

README.md CHANGED Viewed

@@ -1,7 +1,7 @@
 ---
 base_model:
-- denisman/llama-2.4
-- denisman/llama-4.25-k-11
 library_name: transformers
 tags:
 - mergekit
@@ -20,8 +20,8 @@ This model was merged using the SLERP merge method.
 ### Models Merged
 The following models were included in the merge:
-* [denisman/llama-2.4](https://huggingface.co/denisman/llama-2.4)
-* [denisman/llama-4.25-k-11](https://huggingface.co/denisman/llama-4.25-k-11)
 ### Configuration
@@ -32,12 +32,12 @@ The following YAML configuration was used to produce this model:
 slices:
   - sources:
-      - model: denisman/llama-4.25-k-11
         layer_range: [0, 48]
-      - model: denisman/llama-2.4
         layer_range: [0, 48]
 merge_method: slerp
-base_model: denisman/llama-2.4
 parameters:
   t:
     - filter: self_attn

 ---
 base_model:
+- 0x0grandpa0/melancholyson2
+- 0x0grandpa0/melancholysdaughter
 library_name: transformers
 tags:
 - mergekit
 ### Models Merged
 The following models were included in the merge:
+* [0x0grandpa0/melancholyson2](https://huggingface.co/0x0grandpa0/melancholyson2)
+* [0x0grandpa0/melancholysdaughter](https://huggingface.co/0x0grandpa0/melancholysdaughter)
 ### Configuration
 slices:
   - sources:
+      - model: 0x0grandpa0/melancholyson2
         layer_range: [0, 48]
+      - model: 0x0grandpa0/melancholysdaughter
         layer_range: [0, 48]
 merge_method: slerp
+base_model: 0x0grandpa0/melancholyson2
 parameters:
   t:
     - filter: self_attn

config.json CHANGED Viewed

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "denisman/llama-2.4",
   "architectures": [
     "LlamaForCausalLM"
   ],

 {
+  "_name_or_path": "0x0grandpa0/melancholyson2",
   "architectures": [
     "LlamaForCausalLM"
   ],

mergekit_config.yml CHANGED Viewed

@@ -2,12 +2,12 @@
 slices:
   - sources:
-      - model: denisman/llama-4.25-k-11
         layer_range: [0, 48]
-      - model: denisman/llama-2.4
         layer_range: [0, 48]
 merge_method: slerp
-base_model: denisman/llama-2.4
 parameters:
   t:
     - filter: self_attn

 slices:
   - sources:
+      - model: 0x0grandpa0/melancholyson2
         layer_range: [0, 48]
+      - model: 0x0grandpa0/melancholysdaughter
         layer_range: [0, 48]
 merge_method: slerp
+base_model: 0x0grandpa0/melancholyson2
 parameters:
   t:
     - filter: self_attn

model-00001-of-00004.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9b31ad67bde27876356d95d98caf08d049b576fb4311a23d18dc109cd9e1881f
 size 4945284816

 version https://git-lfs.github.com/spec/v1
+oid sha256:980d669bb25cbdfb355a88eb0e3986e110b2175c53e2bc623e4094c5a865766c
 size 4945284816

model-00002-of-00004.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5864f1ce460c54e7972f1f5a710ad7f9a63c3cfd384b577a328386b33089dd0c
 size 4934842800

 version https://git-lfs.github.com/spec/v1
+oid sha256:e0fe545139a3aca0683a8a6368597ec2efd933bf0bd4e3b383a8b4dc20d0b765
 size 4934842800

model-00003-of-00004.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2afe7d4a9c59eec1e9af96e8f4d9ac74e1500032ba373ad31bc4e5b1e472d122
 size 4972600080

 version https://git-lfs.github.com/spec/v1
+oid sha256:2fb0ea5479463939ac4975da99cb454187a54685ca5c1f27537327aff3825722
 size 4972600080

model-00004-of-00004.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:60ef0d95e566d8bed6be8e0f039136488160adb5aba43d249fbba17c20a979db
 size 2806137296

 version https://git-lfs.github.com/spec/v1
+oid sha256:e2ce3b454e565f50c226347cafc4740529550828ba8e338627ddbfbd3ec49631
 size 2806137296