Instructions to use utter-project/EuroLLM-1.7B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use utter-project/EuroLLM-1.7B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="utter-project/EuroLLM-1.7B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("utter-project/EuroLLM-1.7B-Instruct")
model = AutoModelForCausalLM.from_pretrained("utter-project/EuroLLM-1.7B-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use utter-project/EuroLLM-1.7B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "utter-project/EuroLLM-1.7B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "utter-project/EuroLLM-1.7B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/utter-project/EuroLLM-1.7B-Instruct

SGLang

How to use utter-project/EuroLLM-1.7B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "utter-project/EuroLLM-1.7B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "utter-project/EuroLLM-1.7B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "utter-project/EuroLLM-1.7B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "utter-project/EuroLLM-1.7B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use utter-project/EuroLLM-1.7B-Instruct with Docker Model Runner:
```
docker model run hf.co/utter-project/EuroLLM-1.7B-Instruct
```

phmartins commited on Aug 6, 2024

Commit

5025b32

verified ·

1 Parent(s): cc21e79

Update README.md

Browse files

Files changed (1) hide show

README.md +13 -13

README.md CHANGED Viewed

@@ -37,10 +37,10 @@ language:
 - ar
 - hi
 ---
-# Model Card for EuroLLM-1.7B
-This is the model card for the first instruction tuned model of the EuroLLM series: EuroLLM-1.7B-EuroBlocks-v0.1. You can also check the pre-trained version: [EuroLLM-1.7B](https://huggingface.co/Unbabel/EuroLLM-1.7B).
 - **Developed by:** Unbabel, Instituto Superior Técnico, University of Edinburgh, CentraleSupélec University of Paris-Saclay.
 - **Funded by:** European Union.
@@ -52,7 +52,7 @@ This is the model card for the first instruction tuned model of the EuroLLM seri
 The EuroLLM project has the goal of creating a suite of LLMs capable of understanding and generating text in all European Union languages as well as some additional relevant languages.
 EuroLLM-1.7B is a 1.7B parameter model trained on 4 trillion tokens divided across the considered languages and several data sources: Web data, parallel data (en-xx and xx-en), and high-quality datasets.
-EuroLLM-1.7B-EuroBlocks-v0.1 was further instruction tuned on EuroBlocks-v0.1, an instruction tuning dataset predominantly focusing on machine translation.
 ### Model Description
@@ -101,31 +101,31 @@ Here is a summary of the model hyper-parameters:
 ### Machine Translation
-We evaluate EuroLLM-1.7B-EuroBlocks-v0.1 on several machine translation benchmarks: FLORES-200, WMT-23, and WMT-24 comparing it with [Gemma-2B](https://huggingface.co/google/gemma-2b) and [Gemma-7B](https://huggingface.co/google/gemma-7b) (also instruction tuned on EuroBlocks-v0.1).
 The results show that EuroLLM-1.7B is substantially better than Gemma-2B in Machine Translation and competitive with Gemma-7B.
 #### Flores-200
 | Model                          | AVG  | AVG en-xx | AVG xx-en | en-ar | en-bg | en-ca | en-cs | en-da | en-de | en-el | en-es-latam | en-et | en-fi | en-fr | en-ga | en-gl | en-hi | en-hr | en-hu | en-it | en-ja | en-ko | en-lt | en-lv | en-mt | en-nl | en-no | en-pl | en-pt-br | en-ro | en-ru | en-sk | en-sl | en-sv | en-tr | en-uk | en-zh-cn | ar-en | bg-en | ca-en | cs-en | da-en | de-en | el-en | es-latam-en | et-en | fi-en | fr-en | ga-en | gl-en | hi-en | hr-en | hu-en | it-en | ja-en | ko-en | lt-en | lv-en | mt-en | nl-en | no-en | pl-en | pt-br-en | ro-en | ru-en | sk-en | sl-en | sv-en | tr-en | uk-en | zh-cn-en |
 |--------------------------------|------|-----------|-----------|-------|-------|-------|-------|-------|-------|-------|--------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|----------|-------|-------|-------|-------|-------|-------|-------|----------|-------|-------|-------|-------|-------|-------|-------|--------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|----------|-------|-------|-------|-------|-------|-------|-------|----------|
-| EuroLLM-1B-EuroBlocks-v0.1     | 86.10| 85.53     | 86.67     | 83.87 | 88.36 | 84.42 | 88.34 | 88.77 | 86.63 | 86.71 | 85.99        | 86.98 | 87.13 | 87.21 | 72.25 | 85.97 | 74.78 | 82.96 | 85.51 | 87.77 | 89.26 | 86.27 | 86.31 | 86.22 | 67.38 | 86.95 | 88.68 | 87.38   | 89.13 | 88.39 | 87.47 | 87.51 | 85.32 | 89.20 | 86.24 | 86.33   | 86.17 | 85.80 | 87.20 | 87.53 | 87.53 | 89.26 | 88.71 | 86.49        | 86.55 | 87.60 | 88.17 | 88.90 | 79.89 | 87.59 | 87.53 | 86.10 | 86.34 | 87.54 | 86.25 | 86.08 | 85.03 | 85.60 | 78.16 | 86.80 | 89.96   | 85.24 | 88.85 | 88.42 | 85.86 | 87.17 | 86.36 | 89.48   | 86.76 | 86.06 | 85.88 |
-| Gemma-2B-EuroBlocks-v0.1       | 81.56| 78.93     | 84.18     | 75.25 | 82.46 | 83.17 | 82.17 | 84.40 | 83.20 | 79.63 | 84.15        | 72.63 | 81.00 | 85.12 | 38.79 | 82.00 | 67.00 | 81.18 | 78.24 | 84.80 | 87.08 | 82.04 | 73.02 | 68.41 | 56.67 | 83.30 | 86.69 | 83.07   | 86.82 | 84.00 | 84.55 | 77.93 | 76.19 | 80.77 | 79.76 | 84.19   | 84.10 | 83.67 | 85.73 | 86.89 | 86.38 | 88.39 | 88.11 | 84.68        | 86.11 | 83.45 | 86.45 | 88.22 | 50.88 | 86.44 | 85.87 | 85.33 | 85.16 | 86.75 | 85.62 | 85.00 | 81.55 | 81.45 | 67.90 | 85.95 | 89.05   | 84.18 | 88.27 | 87.38 | 85.13 | 85.22 | 83.86 | 87.83   | 84.96 | 85.15 | 85.10 |
-| Gemma-7B-EuroBlocks-v0.1       | 86.16| 85.49     | 86.82     | 83.39 | 88.32 | 85.82 | 88.88 | 89.01 | 86.96 | 86.62 | 86.31        | 84.42 | 88.11 | 87.46 | 61.85 | 86.10 | 77.91 | 87.01 | 85.81 | 87.57 | 89.88 | 87.24 | 84.47 | 83.15 | 67.13 | 86.50 | 90.44 | 87.57   | 89.22 | 89.13 | 88.58 | 86.73 | 84.68 | 88.16 | 86.87 | 88.40   | 87.11 | 86.65 | 87.25 | 88.17 | 87.47 | 89.59 | 88.44 | 86.76        | 86.66 | 87.55 | 88.88 | 88.86 | 73.46 | 87.63 | 88.43 | 87.12 | 87.31 | 87.49 | 87.20 | 87.15 | 85.16 | 85.96 | 78.39 | 86.73 | 90.52   | 85.38 | 89.17 | 88.75 | 86.35 | 86.82 | 86.21 | 89.39   | 88.20 | 86.45 | 86.28 |
 #### WMT-23
 | Model                          | AVG  | AVG en-xx | AVG xx-en | AVG xx-xx | en-de | en-cs | en-uk | en-ru | en-zh-cn | de-en | uk-en | ru-en | zh-cn-en | cs-uk |
 |--------------------------------|------|-----------|-----------|-----------|-------|-------|-------|-------|----------|-------|-------|-------|----------|-------|
-| EuroLLM-1.7B-EuroBlocks-v0.1   | 82.56| 82.30     | 82.07     | 85.81     | 80.99 | 84.42 | 80.74 | 81.94 | 83.42    | 83.74 | 85.06 | 81.00 | 78.49    | 85.81 |
-| Gemma-2B-EuroBlocks-v0.1       | 79.86| 78.35     | 81.32     | 81.56     | 76.54 | 76.35 | 77.62 | 78.88 | 82.36    | 82.85 | 83.83 | 80.17 | 78.42    | 81.56 |
-| Gemma-7B-EuroBlocks-v0.1       | 83.90| 83.70     | 83.21     | 87.61     | 82.15 | 84.68 | 83.05 | 83.85 | 84.79    | 84.40 | 85.86 | 82.55 | 80.01    | 87.61 |
 #### WMT-24
 | Model | AVG | AVG en-xx | AVG xx-xx | en-es-latam | en-cs | en-ru | en-uk | en-ja | en-zh-cn | en-hi | cs-uk | ja-zh-cn |
 |---------|------|------|-------|-------|-------|-------|--------|--------|-------|-------|-------|-----|
-| EuroLLM-1.7B-EuroBlocks-v0.1| 78.45|78.65|77.67|79.05|80.93|80.33|78.05|78.72|81.87|80.15|70.10|82.65|72.69|
-|Gemma-2B-EuroBlocks-v0.1| 74.71|74.25|76.57|75.21|78.84|70.40|74.44|75.55|78.32|78.70|62.51|79.97|73.17|
-|Gemma-7B-EuroBlocks-v0.1| 80.88|80.45|82.60|80.43|81.91|80.14|80.32|82.17|84.08|81.86|72.71|85.55|79.65|
 ### General Benchmarks
 We also compare EuroLLM-1.7B with [TinyLlama-1.1-3T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T) and [Gemma-2B](https://huggingface.co/google/gemma-2b) on 3 general benchmarks: Arc Challenge, Hellaswag, and MMLU.

 - ar
 - hi
 ---
+# Model Card for EuroLLM-1.7B-Instruct
+This is the model card for the first instruction tuned model of the EuroLLM series: EuroLLM-1.7B-Instruct. You can also check the pre-trained version: [EuroLLM-1.7B](https://huggingface.co/utter-project/EuroLLM-1.7B).
 - **Developed by:** Unbabel, Instituto Superior Técnico, University of Edinburgh, CentraleSupélec University of Paris-Saclay.
 - **Funded by:** European Union.
 The EuroLLM project has the goal of creating a suite of LLMs capable of understanding and generating text in all European Union languages as well as some additional relevant languages.
 EuroLLM-1.7B is a 1.7B parameter model trained on 4 trillion tokens divided across the considered languages and several data sources: Web data, parallel data (en-xx and xx-en), and high-quality datasets.
+EuroLLM-1.7B-Instruct was further instruction tuned on EuroBlocks, a dataset predominantly focusing on machine translation.
 ### Model Description
 ### Machine Translation
+We evaluate EuroLLM-1.7B-Instruct on several machine translation benchmarks: FLORES-200, WMT-23, and WMT-24 comparing it with [Gemma-2B](https://huggingface.co/google/gemma-2b) and [Gemma-7B](https://huggingface.co/google/gemma-7b) (also instruction tuned on EuroBlocks-v0.1).
 The results show that EuroLLM-1.7B is substantially better than Gemma-2B in Machine Translation and competitive with Gemma-7B.
 #### Flores-200
 | Model                          | AVG  | AVG en-xx | AVG xx-en | en-ar | en-bg | en-ca | en-cs | en-da | en-de | en-el | en-es-latam | en-et | en-fi | en-fr | en-ga | en-gl | en-hi | en-hr | en-hu | en-it | en-ja | en-ko | en-lt | en-lv | en-mt | en-nl | en-no | en-pl | en-pt-br | en-ro | en-ru | en-sk | en-sl | en-sv | en-tr | en-uk | en-zh-cn | ar-en | bg-en | ca-en | cs-en | da-en | de-en | el-en | es-latam-en | et-en | fi-en | fr-en | ga-en | gl-en | hi-en | hr-en | hu-en | it-en | ja-en | ko-en | lt-en | lv-en | mt-en | nl-en | no-en | pl-en | pt-br-en | ro-en | ru-en | sk-en | sl-en | sv-en | tr-en | uk-en | zh-cn-en |
 |--------------------------------|------|-----------|-----------|-------|-------|-------|-------|-------|-------|-------|--------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|----------|-------|-------|-------|-------|-------|-------|-------|----------|-------|-------|-------|-------|-------|-------|-------|--------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|----------|-------|-------|-------|-------|-------|-------|-------|----------|
+| EuroLLM-1.7B-Instruct          | 86.10| 85.53     | 86.67     | 83.87 | 88.36 | 84.42 | 88.34 | 88.77 | 86.63 | 86.71 | 85.99        | 86.98 | 87.13 | 87.21 | 72.25 | 85.97 | 74.78 | 82.96 | 85.51 | 87.77 | 89.26 | 86.27 | 86.31 | 86.22 | 67.38 | 86.95 | 88.68 | 87.38   | 89.13 | 88.39 | 87.47 | 87.51 | 85.32 | 89.20 | 86.24 | 86.33   | 86.17 | 85.80 | 87.20 | 87.53 | 87.53 | 89.26 | 88.71 | 86.49        | 86.55 | 87.60 | 88.17 | 88.90 | 79.89 | 87.59 | 87.53 | 86.10 | 86.34 | 87.54 | 86.25 | 86.08 | 85.03 | 85.60 | 78.16 | 86.80 | 89.96   | 85.24 | 88.85 | 88.42 | 85.86 | 87.17 | 86.36 | 89.48   | 86.76 | 86.06 | 85.88 |
+| Gemma-2B-EuroBlocks       | 81.56| 78.93     | 84.18     | 75.25 | 82.46 | 83.17 | 82.17 | 84.40 | 83.20 | 79.63 | 84.15        | 72.63 | 81.00 | 85.12 | 38.79 | 82.00 | 67.00 | 81.18 | 78.24 | 84.80 | 87.08 | 82.04 | 73.02 | 68.41 | 56.67 | 83.30 | 86.69 | 83.07   | 86.82 | 84.00 | 84.55 | 77.93 | 76.19 | 80.77 | 79.76 | 84.19   | 84.10 | 83.67 | 85.73 | 86.89 | 86.38 | 88.39 | 88.11 | 84.68        | 86.11 | 83.45 | 86.45 | 88.22 | 50.88 | 86.44 | 85.87 | 85.33 | 85.16 | 86.75 | 85.62 | 85.00 | 81.55 | 81.45 | 67.90 | 85.95 | 89.05   | 84.18 | 88.27 | 87.38 | 85.13 | 85.22 | 83.86 | 87.83   | 84.96 | 85.15 | 85.10 |
+| Gemma-7B-EuroBlocks       | 86.16| 85.49     | 86.82     | 83.39 | 88.32 | 85.82 | 88.88 | 89.01 | 86.96 | 86.62 | 86.31        | 84.42 | 88.11 | 87.46 | 61.85 | 86.10 | 77.91 | 87.01 | 85.81 | 87.57 | 89.88 | 87.24 | 84.47 | 83.15 | 67.13 | 86.50 | 90.44 | 87.57   | 89.22 | 89.13 | 88.58 | 86.73 | 84.68 | 88.16 | 86.87 | 88.40   | 87.11 | 86.65 | 87.25 | 88.17 | 87.47 | 89.59 | 88.44 | 86.76        | 86.66 | 87.55 | 88.88 | 88.86 | 73.46 | 87.63 | 88.43 | 87.12 | 87.31 | 87.49 | 87.20 | 87.15 | 85.16 | 85.96 | 78.39 | 86.73 | 90.52   | 85.38 | 89.17 | 88.75 | 86.35 | 86.82 | 86.21 | 89.39   | 88.20 | 86.45 | 86.28 |
 #### WMT-23
 | Model                          | AVG  | AVG en-xx | AVG xx-en | AVG xx-xx | en-de | en-cs | en-uk | en-ru | en-zh-cn | de-en | uk-en | ru-en | zh-cn-en | cs-uk |
 |--------------------------------|------|-----------|-----------|-----------|-------|-------|-------|-------|----------|-------|-------|-------|----------|-------|
+| EuroLLM-1.7B-Instruct          | 82.56| 82.30     | 82.07     | 85.81     | 80.99 | 84.42 | 80.74 | 81.94 | 83.42    | 83.74 | 85.06 | 81.00 | 78.49    | 85.81 |
+| Gemma-2B-EuroBlocks       | 79.86| 78.35     | 81.32     | 81.56     | 76.54 | 76.35 | 77.62 | 78.88 | 82.36    | 82.85 | 83.83 | 80.17 | 78.42    | 81.56 |
+| Gemma-7B-EuroBlocks       | 83.90| 83.70     | 83.21     | 87.61     | 82.15 | 84.68 | 83.05 | 83.85 | 84.79    | 84.40 | 85.86 | 82.55 | 80.01    | 87.61 |
 #### WMT-24
 | Model | AVG | AVG en-xx | AVG xx-xx | en-es-latam | en-cs | en-ru | en-uk | en-ja | en-zh-cn | en-hi | cs-uk | ja-zh-cn |
 |---------|------|------|-------|-------|-------|-------|--------|--------|-------|-------|-------|-----|
+| EuroLLM-1.7B-Instruct| 78.45|78.65|77.67|79.05|80.93|80.33|78.05|78.72|81.87|80.15|70.10|82.65|72.69|
+|Gemma-2B-EuroBlocks  | 74.71|74.25|76.57|75.21|78.84|70.40|74.44|75.55|78.32|78.70|62.51|79.97|73.17|
+|Gemma-7B-EuroBlocks   | 80.88|80.45|82.60|80.43|81.91|80.14|80.32|82.17|84.08|81.86|72.71|85.55|79.65|
 ### General Benchmarks
 We also compare EuroLLM-1.7B with [TinyLlama-1.1-3T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T) and [Gemma-2B](https://huggingface.co/google/gemma-2b) on 3 general benchmarks: Arc Challenge, Hellaswag, and MMLU.