Instructions to use NexaAI/octopus-planning with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use NexaAI/octopus-planning with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="NexaAI/octopus-planning", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("NexaAI/octopus-planning", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("NexaAI/octopus-planning", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use NexaAI/octopus-planning with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NexaAI/octopus-planning"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NexaAI/octopus-planning",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/NexaAI/octopus-planning

SGLang

How to use NexaAI/octopus-planning with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "NexaAI/octopus-planning" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NexaAI/octopus-planning",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "NexaAI/octopus-planning" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NexaAI/octopus-planning",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use NexaAI/octopus-planning with Docker Model Runner:
```
docker model run hf.co/NexaAI/octopus-planning
```

octopus-planning

Commit History

Upload Phi3ForCausalLM

8f258f7
verified

zackli4ai commited on Jun 27, 2024

Upload tokenizer

19d93c1
verified

zackli4ai commited on Jun 27, 2024

Upload Phi3ForCausalLM

c119fb7
verified

zackli4ai commited on Jun 27, 2024

Update special_tokens_map.json

ff2a91e
verified

zackli4ai commited on Jun 27, 2024

Update README.md

7d9d6c6
verified

zackli4ai commited on Jun 27, 2024

Update README.md

077b72e
verified

zackli4ai commited on Jun 27, 2024

Upload Phi3ForCausalLM

ab7bac5
verified

zackli4ai commited on Jun 26, 2024

Upload tokenizer

5bc75be
verified

zackli4ai commited on Jun 26, 2024

wip

f8e536d

Zack Zhiyuan Li commited on Jun 26, 2024

Upload 1-demo.png

7724aff
verified

zackli4ai commited on Jun 25, 2024

Upload 1-demo.png

df9d985
verified

zackli4ai commited on Jun 25, 2024

Upload GemmaForCausalLM

1b6f463
verified

zackli4ai commited on Jun 25, 2024

Upload tokenizer

0385f0f
verified

zackli4ai commited on Jun 25, 2024

update

6e1d240

Zack Zhiyuan Li commited on Jun 24, 2024

Merge branch 'main' of https://huggingface.co/NexaAIDev/octopus-planning

fc231d4

Zack Zhiyuan Li commited on Jun 24, 2024

update inference code

0efe51a

Zack Zhiyuan Li commited on Jun 24, 2024

Upload Phi3ForCausalLM

3f2016c
verified

zackli4ai commited on Jun 24, 2024

Upload tokenizer

47091d0
verified

zackli4ai commited on Jun 24, 2024

add photo to readme

218a115

Zack Zhiyuan Li commited on Jun 24, 2024

Upload Phi3ForCausalLM

965ff9d
verified

zackli4ai commited on Jun 24, 2024

Upload 1-demo.png

a1adde1
verified

zackli4ai commited on Jun 24, 2024

update README.md

6060ae8

Zack Zhiyuan Li commited on Jun 24, 2024

initial commit

39ae8bf
verified

zackli4ai commited on Jun 24, 2024

Commit History

Upload Phi3ForCausalLM 8f258f7 verified

Upload tokenizer 19d93c1 verified

Upload Phi3ForCausalLM c119fb7 verified

Update special_tokens_map.json ff2a91e verified

Update README.md 7d9d6c6 verified

Update README.md 077b72e verified

Upload Phi3ForCausalLM ab7bac5 verified

Upload tokenizer 5bc75be verified

wip f8e536d

Upload 1-demo.png 7724aff verified

Upload 1-demo.png df9d985 verified

Upload GemmaForCausalLM 1b6f463 verified

Upload tokenizer 0385f0f verified

update 6e1d240

Merge branch 'main' of https://huggingface.co/NexaAIDev/octopus-planning fc231d4

update inference code 0efe51a

Upload Phi3ForCausalLM 3f2016c verified

Upload tokenizer 47091d0 verified

add photo to readme 218a115

Upload Phi3ForCausalLM 965ff9d verified

Upload 1-demo.png a1adde1 verified

update README.md 6060ae8

initial commit 39ae8bf verified

Upload Phi3ForCausalLM

8f258f7
verified

Upload tokenizer

19d93c1
verified

Upload Phi3ForCausalLM

c119fb7
verified

Update special_tokens_map.json

ff2a91e
verified

Update README.md

7d9d6c6
verified

Update README.md

077b72e
verified

Upload Phi3ForCausalLM

ab7bac5
verified

Upload tokenizer

5bc75be
verified

wip

f8e536d

Upload 1-demo.png

7724aff
verified

Upload 1-demo.png

df9d985
verified

Upload GemmaForCausalLM

1b6f463
verified

Upload tokenizer

0385f0f
verified

update

6e1d240

Merge branch 'main' of https://huggingface.co/NexaAIDev/octopus-planning

fc231d4

update inference code

0efe51a

Upload Phi3ForCausalLM

3f2016c
verified

Upload tokenizer

47091d0
verified

add photo to readme

218a115

Upload Phi3ForCausalLM

965ff9d
verified

Upload 1-demo.png

a1adde1
verified

update README.md

6060ae8

initial commit

39ae8bf
verified