Instructions to use Deepthoughtworks/gpt-neo-2.7B__low-cpu with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Deepthoughtworks/gpt-neo-2.7B__low-cpu with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Deepthoughtworks/gpt-neo-2.7B__low-cpu")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Deepthoughtworks/gpt-neo-2.7B__low-cpu")
model = AutoModelForCausalLM.from_pretrained("Deepthoughtworks/gpt-neo-2.7B__low-cpu")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Deepthoughtworks/gpt-neo-2.7B__low-cpu with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Deepthoughtworks/gpt-neo-2.7B__low-cpu"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Deepthoughtworks/gpt-neo-2.7B__low-cpu",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Deepthoughtworks/gpt-neo-2.7B__low-cpu

SGLang

How to use Deepthoughtworks/gpt-neo-2.7B__low-cpu with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Deepthoughtworks/gpt-neo-2.7B__low-cpu" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Deepthoughtworks/gpt-neo-2.7B__low-cpu",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Deepthoughtworks/gpt-neo-2.7B__low-cpu" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Deepthoughtworks/gpt-neo-2.7B__low-cpu",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Deepthoughtworks/gpt-neo-2.7B__low-cpu with Docker Model Runner:
```
docker model run hf.co/Deepthoughtworks/gpt-neo-2.7B__low-cpu
```

gpt-neo-2.7B__low-cpu

Commit History

Update requirements.txt

2d9bf83

fwittel commited on Nov 19, 2022

Create requirements.txt

79fc1d4

fwittel commited on Nov 19, 2022

Update handler.py (#1)

9bd5173

fwittel

philschmid commited on Nov 19, 2022

Switch to AutoModelForSeq2SeqLM

0af1b4a

fwittel commited on Nov 18, 2022

Delete .DS_Store

9ff7017

fwittel commited on Nov 18, 2022

switch to AutoModelForCausalLM

22bc6be

fwittel commited on Nov 18, 2022

Remove cache, requirements.txt

d4725fd

fwittel commited on Nov 18, 2022

Removed requirements

860fc27

fwittel commited on Nov 17, 2022

Added device-selection to handler.py

3a31a5b

fwittel commited on Nov 17, 2022

Added tokenizer to handler.py

040e104

fwittel commited on Nov 17, 2022

added requirements

4c61288

fwittel commited on Nov 15, 2022

added custom handler

ba13b5a

fwittel commited on Nov 15, 2022

Fix invalid readma

97ce3d1

fwittel commited on Nov 14, 2022

Update README.md

51568a6

stellaathena commited on Dec 31, 2021

Updated tags to correctly link with the Pile

5e755b1

stellaathena commited on Dec 31, 2021

Update README.md

6f23148

stellaathena commited on Sep 11, 2021

Update README.md

88f8889

stellaathena commited on Sep 11, 2021

add flax model

0b8087b

valhalla commited on Jul 4, 2021

Update README.md

b41a392

lg commited on May 21, 2021

Updated citation info

1172dff

stellaathena commited on May 18, 2021

Updated LFS tracked files

df3bd66

guillaume-be commited on May 6, 2021

Addition of Rust model

9b4ecbc

guillaume-be commited on May 5, 2021

Update README.md

3e1c92c

stellaathena commited on Apr 12, 2021

Update model max length

5640038

lysandre HF Staff commited on Apr 6, 2021

Update README.md

9130025

stellaathena commited on Apr 4, 2021

Update README.md

058e8e2

stellaathena commited on Mar 31, 2021

Update README.md

40fb054

stellaathena commited on Mar 31, 2021

Update config.json

f5e76ba

valhalla commited on Mar 31, 2021

Sample

5957e80

LysandreJik commited on Mar 30, 2021

Sample

7dc5d8d

LysandreJik commited on Mar 30, 2021

Add model card

5813d1e

LysandreJik commited on Mar 30, 2021

add files

ef4a5f2

valhalla commited on Mar 30, 2021

initial commit

d31f415

system HF Staff commited on Mar 30, 2021

Commit History

Update requirements.txt 2d9bf83

Create requirements.txt 79fc1d4

Update handler.py (#1) 9bd5173

Switch to AutoModelForSeq2SeqLM 0af1b4a

Delete .DS_Store 9ff7017

switch to AutoModelForCausalLM 22bc6be

Remove cache, requirements.txt d4725fd

Removed requirements 860fc27

Added device-selection to handler.py 3a31a5b

Added tokenizer to handler.py 040e104

added requirements 4c61288

added custom handler ba13b5a

Fix invalid readma 97ce3d1

Update README.md 51568a6

Updated tags to correctly link with the Pile 5e755b1

Update README.md 6f23148

Update README.md 88f8889

add flax model 0b8087b

Update README.md b41a392

Updated citation info 1172dff

Updated LFS tracked files df3bd66

Addition of Rust model 9b4ecbc

Update README.md 3e1c92c

Update model max length 5640038

Update README.md 9130025

Update README.md 058e8e2

Update README.md 40fb054

Update config.json f5e76ba

Sample 5957e80

Sample 7dc5d8d

Add model card 5813d1e

add files ef4a5f2

initial commit d31f415

Update requirements.txt

2d9bf83

Create requirements.txt

79fc1d4

Update handler.py (#1)

9bd5173

Switch to AutoModelForSeq2SeqLM

0af1b4a

Delete .DS_Store

9ff7017

switch to AutoModelForCausalLM

22bc6be

Remove cache, requirements.txt

d4725fd

Removed requirements

860fc27

Added device-selection to handler.py

3a31a5b

Added tokenizer to handler.py

040e104

added requirements

4c61288

added custom handler

ba13b5a

Fix invalid readma

97ce3d1

Update README.md

51568a6

Updated tags to correctly link with the Pile

5e755b1

Update README.md

6f23148

Update README.md

88f8889

add flax model

0b8087b

Update README.md

b41a392

Updated citation info

1172dff

Updated LFS tracked files

df3bd66

Addition of Rust model

9b4ecbc

Update README.md

3e1c92c

Update model max length

5640038

Update README.md

9130025

Update README.md

058e8e2

Update README.md

40fb054

Update config.json

f5e76ba

Sample

5957e80

Sample

7dc5d8d

Add model card

5813d1e

add files

ef4a5f2

initial commit

d31f415