Instructions to use OEvortex/HelpingAI-180B-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OEvortex/HelpingAI-180B-base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OEvortex/HelpingAI-180B-base")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OEvortex/HelpingAI-180B-base")
model = AutoModelForCausalLM.from_pretrained("OEvortex/HelpingAI-180B-base", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OEvortex/HelpingAI-180B-base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OEvortex/HelpingAI-180B-base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OEvortex/HelpingAI-180B-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/OEvortex/HelpingAI-180B-base

SGLang

How to use OEvortex/HelpingAI-180B-base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OEvortex/HelpingAI-180B-base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OEvortex/HelpingAI-180B-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OEvortex/HelpingAI-180B-base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OEvortex/HelpingAI-180B-base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use OEvortex/HelpingAI-180B-base with Docker Model Runner:
```
docker model run hf.co/OEvortex/HelpingAI-180B-base
```

HelpingAI-180B-base

Commit History

Apr 5, 2024

10:01
Adding `safetensors` variant of this model⁠

Abhaykoul

Verified signature
b8b6f0c
07:13
Update README.md

Abhaykoul

Verified signature
5648e40
07:09
Create README.md

Abhaykoul

Verified signature
49b8306
06:56
Rename tokenizer_config (1).json to tokenizer_config.json

Abhaykoul

Verified signature
3dda717
06:56
gemma tokenizer

Abhaykoul

Verified signature
2fb91fa
06:55
Upload 3 files

Abhaykoul

Verified signature
dde1565
06:55
Delete tokenizer_config.json

Abhaykoul

Verified signature
edc3008
06:54
Delete tokenizer.json

Abhaykoul

Verified signature
17b675b
06:53
Upload 2 files

Abhaykoul

Verified signature
ff67d08
06:52
Update generation_config.json

Abhaykoul

Verified signature
3e1cb7d
06:50
Upload generation_config.json

Abhaykoul

Verified signature
f575891
06:45
Upload folder using huggingface_hub

Abhaykoul

Verified signature
d6386ec
06:35
Upload folder using huggingface_hub

Abhaykoul

Verified signature
a768708
06:17
Upload folder using huggingface_hub

Abhaykoul

Verified signature
5862ef5
06:02
Upload folder using huggingface_hub

Abhaykoul

Verified signature
346e076
05:46
Upload folder using huggingface_hub

Abhaykoul

Verified signature
d6b611f
05:31
Upload folder using huggingface_hub

Abhaykoul

Verified signature
b347ab2
05:15
Upload folder using huggingface_hub

Abhaykoul

Verified signature
c01e8dd
05:00
Upload folder using huggingface_hub

Abhaykoul

Verified signature
712eb4b
04:44
Upload folder using huggingface_hub

Abhaykoul

Verified signature
ac49c09
04:24
Upload folder using huggingface_hub

Abhaykoul

Verified signature
7332b8d
04:09
Upload folder using huggingface_hub

Abhaykoul

Verified signature
cadaeeb

Apr 4, 2024

15:21
Upload folder using huggingface_hub

Abhaykoul

Verified signature
a1c5b21
15:06
Upload folder using huggingface_hub

Abhaykoul

Verified signature
fe3ee61
14:50
Upload folder using huggingface_hub

Abhaykoul

Verified signature
5e4ac92
14:37
Upload folder using huggingface_hub

Abhaykoul

Verified signature
8bb4a14
14:23
Upload folder using huggingface_hub

Abhaykoul

Verified signature
6441b41
14:08
Upload folder using huggingface_hub

Abhaykoul

Verified signature
526d265
13:52
Upload folder using huggingface_hub

Abhaykoul

Verified signature
9b4019c
13:39
Upload folder using huggingface_hub

Abhaykoul

Verified signature
1f307d3
13:24
Upload folder using huggingface_hub

Abhaykoul

Verified signature
b9fd0a5
13:10
Upload folder using huggingface_hub

Abhaykoul

Verified signature
b6874cf
12:56
Upload folder using huggingface_hub

Abhaykoul

Verified signature
2ea6bb0
12:42
Upload folder using huggingface_hub

Abhaykoul

Verified signature
3b299bb
12:28
Upload folder using huggingface_hub

Abhaykoul

Verified signature
2c06356
12:13
Upload folder using huggingface_hub

Abhaykoul

Verified signature
e35590d
11:59
Upload folder using huggingface_hub

Abhaykoul

Verified signature
825d131
11:45
Upload folder using huggingface_hub

Abhaykoul

Verified signature
d88b785
11:30
Upload folder using huggingface_hub

Abhaykoul

Verified signature
7beddbe
11:15
Upload folder using huggingface_hub

Abhaykoul

Verified signature
388979b
11:00
Upload folder using huggingface_hub

Abhaykoul

Verified signature
81f93b3
10:46
Upload folder using huggingface_hub

Abhaykoul

Verified signature
c289090
10:31
Upload folder using huggingface_hub

Abhaykoul

Verified signature
49a8ed6
10:17
Upload folder using huggingface_hub

Abhaykoul

Verified signature
2b04f44
10:08
Create config.json

Abhaykoul

Verified signature
efd9e9f
10:01
Upload 2 files

Abhaykoul

Verified signature
95b87ff
09:58
Using Bloom tokenizer for this model

Abhaykoul

Verified signature
4c3fd57
09:50
Upload folder using huggingface_hub

Abhaykoul

Verified signature
dfcc7f9
09:37
Upload folder using huggingface_hub

Abhaykoul

Verified signature
caa899f
09:21
Upload folder using huggingface_hub

Abhaykoul

Verified signature
10cc4e9

Commit History

Apr 5, 2024

Adding `safetensors` variant of this model⁠

Update README.md

Create README.md

Rename tokenizer_config (1).json to tokenizer_config.json

gemma tokenizer

Upload 3 files

Delete tokenizer_config.json

Delete tokenizer.json

Upload 2 files

Update generation_config.json

Upload generation_config.json

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Apr 4, 2024

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Create config.json

Upload 2 files

Using Bloom tokenizer for this model

Upload folder using huggingface_hub

Upload folder using huggingface_hub

Upload folder using huggingface_hub