Instructions to use nuprl/MultiPL-T-StarCoderBase_15b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nuprl/MultiPL-T-StarCoderBase_15b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nuprl/MultiPL-T-StarCoderBase_15b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nuprl/MultiPL-T-StarCoderBase_15b")
model = AutoModelForCausalLM.from_pretrained("nuprl/MultiPL-T-StarCoderBase_15b")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use nuprl/MultiPL-T-StarCoderBase_15b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nuprl/MultiPL-T-StarCoderBase_15b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nuprl/MultiPL-T-StarCoderBase_15b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/nuprl/MultiPL-T-StarCoderBase_15b

SGLang

How to use nuprl/MultiPL-T-StarCoderBase_15b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nuprl/MultiPL-T-StarCoderBase_15b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nuprl/MultiPL-T-StarCoderBase_15b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nuprl/MultiPL-T-StarCoderBase_15b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nuprl/MultiPL-T-StarCoderBase_15b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use nuprl/MultiPL-T-StarCoderBase_15b with Docker Model Runner:
```
docker model run hf.co/nuprl/MultiPL-T-StarCoderBase_15b
```

cassanof commited on Aug 16, 2023

Commit

1d70af8

1 Parent(s): c1f985d

model_starcoder15b_all_lang_equal_paper-epoch7

Browse files

Files changed (5) hide show

config.json +1 -1
pytorch_model-00001-of-00004.bin +1 -1
pytorch_model-00002-of-00004.bin +1 -1
pytorch_model-00003-of-00004.bin +1 -1
pytorch_model-00004-of-00004.bin +1 -1

config.json CHANGED Viewed

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "model_starcoder15b_all_lang_equal_paper/checkpoint-1560",
   "activation_function": "gelu",
   "architectures": [
     "GPTBigCodeForCausalLM"

 {
+  "_name_or_path": "model_starcoder15b_all_lang_equal_paper/checkpoint-1820",
   "activation_function": "gelu",
   "architectures": [
     "GPTBigCodeForCausalLM"

pytorch_model-00001-of-00004.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b8de572f5aa0a4fdd90fcdb5b5a42036f74c9aa41a17c02f2dd3939d6f3091f7
 size 9957983029

 version https://git-lfs.github.com/spec/v1
+oid sha256:5dacd3ac9c10999b6f950d3713e3902c2d1bc12ae5ad4a1d17cb209de5a19969
 size 9957983029

pytorch_model-00002-of-00004.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:88ed699ec7172dd58746f8621eed377c52ae60ef795e140dba20a2b6fb89b2fb
 size 9857381671

 version https://git-lfs.github.com/spec/v1
+oid sha256:faec3159bdc81ce402358bc574af40cc8730827c0922889bc68abcc17361c2bb
 size 9857381671

pytorch_model-00003-of-00004.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:509a0d4c80ee7a7ccde1bc1ede470161e583672fb1ef3d583b9b1ecbc2128efc
 size 9857381671

 version https://git-lfs.github.com/spec/v1
+oid sha256:4f98f19bf7444be2f35442dc8f948428fe7f625a207e40cd077f22c7d142d6de
 size 9857381671

pytorch_model-00004-of-00004.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b745834b6944360a4841e2c97ad6144485625268c07cdf13434cd8601976119a
 size 1362327955

 version https://git-lfs.github.com/spec/v1
+oid sha256:34ecc80ff9bb0285d753b030a64dcb6797bb59bf80e0c04b74c7bbb60dedbd03
 size 1362327955