Instructions to use DAMO-NLP-MT/polylm-multialpaca-13b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DAMO-NLP-MT/polylm-multialpaca-13b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DAMO-NLP-MT/polylm-multialpaca-13b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DAMO-NLP-MT/polylm-multialpaca-13b")
model = AutoModelForCausalLM.from_pretrained("DAMO-NLP-MT/polylm-multialpaca-13b")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use DAMO-NLP-MT/polylm-multialpaca-13b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DAMO-NLP-MT/polylm-multialpaca-13b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DAMO-NLP-MT/polylm-multialpaca-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/DAMO-NLP-MT/polylm-multialpaca-13b

SGLang

How to use DAMO-NLP-MT/polylm-multialpaca-13b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DAMO-NLP-MT/polylm-multialpaca-13b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DAMO-NLP-MT/polylm-multialpaca-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DAMO-NLP-MT/polylm-multialpaca-13b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DAMO-NLP-MT/polylm-multialpaca-13b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use DAMO-NLP-MT/polylm-multialpaca-13b with Docker Model Runner:
```
docker model run hf.co/DAMO-NLP-MT/polylm-multialpaca-13b
```

Model Card for PolyLM-Multialpaca

This model is finetuned on polyLM-13b using multialpaca (a self-instruction dataset)

Demo

Open

Bias, Risks, and Limitations

The information below in this section are copied from the model's official model card:

Our contributions are fully methodological: adding the support of multilingualism to LLM during training and SFT phases. It is unavoidable that PolyLM might exhibit several common deficiencies of language models, e.g. hallucination and toxicity. PolyLM should not be used directly in any application, without a prior assessment of safety and fairness concerns specific to the application.

This version activates the instruction-following capability of PolyLM through self-instruction, but currently, the training instructions are relatively simple and the support for abilities such as multi-turn dialogue, context understanding, CoT, Plugin, etc. is not very friendly. We are making efforts to develop a new version.

Citation

BibTeX:

@misc{wei2023polylm,
    title={PolyLM: An Open Source Polyglot Large Language Model},
    author={Xiangpeng Wei and Haoran Wei and Huan Lin and Tianhao Li and Pei Zhang and Xingzhang Ren and Mei Li and Yu Wan and Zhiwei Cao and Binbin Xie and Tianxiang Hu and Shangjie Li and Binyuan Hui and Bowen Yu and Dayiheng Liu and Baosong Yang and Fei Huang and Jun Xie},
    year={2023},
    eprint={2307.06018},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

Downloads last month: 11

Paper for DAMO-NLP-MT/polylm-multialpaca-13b

PolyLM: An Open Source Polyglot Large Language Model

Paper • 2307.06018 • Published Jul 12, 2023 • 26