Instructions to use FreedomIntelligence/Apollo-72B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FreedomIntelligence/Apollo-72B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FreedomIntelligence/Apollo-72B", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("FreedomIntelligence/Apollo-72B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FreedomIntelligence/Apollo-72B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FreedomIntelligence/Apollo-72B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/Apollo-72B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/FreedomIntelligence/Apollo-72B

SGLang

How to use FreedomIntelligence/Apollo-72B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FreedomIntelligence/Apollo-72B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/Apollo-72B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FreedomIntelligence/Apollo-72B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/Apollo-72B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use FreedomIntelligence/Apollo-72B with Docker Model Runner:
```
docker model run hf.co/FreedomIntelligence/Apollo-72B
```

Apollo-72B

Commit History

Apr 23, 2024

14:40
Upload ./model-00005-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
5afffa1
11:28
Upload result.png

Xidong

Verified signature
ddb5607
11:27
Upload README.md

Xidong

Verified signature
052bef7
11:27
Upload result.png

Xidong

Verified signature
06ad98a
09:00
Upload README.md

Xidong

Verified signature
9fc9abd
08:59
Upload 3 files

Xidong

Verified signature
080fe3a
08:59
Create assets/logo

Xidong

Verified signature
ef3c1a1

Apr 22, 2024

22:44
Upload Apollo-72B/model-00029-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
755290c
22:35
Upload Apollo-72B/model-00003-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
1261842
22:26
Upload Apollo-72B/generation_config.json with huggingface_hub

Xidong

Verified signature
e2fd061
22:26
Upload Apollo-72B/model-00026-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
fa5ecff
22:18
Upload Apollo-72B/model-00011-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
93b0661
22:09
Upload Apollo-72B/model-00014-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
bee2786
22:01
Upload Apollo-72B/model-00004-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
28c98fa
21:52
Upload Apollo-72B/model-00001-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
85d347f
21:44
Upload Apollo-72B/model-00025-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
5542f24
21:35
Upload Apollo-72B/model-00007-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
921e02f
21:26
Upload Apollo-72B/model-00023-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
483ee97
21:18
Upload Apollo-72B/model-00015-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
7526b97
21:08
Upload Apollo-72B/model-00012-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
b813a33
21:00
Upload Apollo-72B/model-00019-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
8e20709
20:51
Upload Apollo-72B/model-00022-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
0fc0345
20:42
Upload Apollo-72B/model-00008-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
91c3018
20:32
Upload Apollo-72B/model-00030-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
c930705
20:26
Upload Apollo-72B/modeling_qwen.py with huggingface_hub

Xidong

Verified signature
01ba51c
20:26
Upload Apollo-72B/special_tokens_map.json with huggingface_hub

Xidong

Verified signature
803d071
20:26
Upload Apollo-72B/configuration_qwen.py with huggingface_hub

Xidong

Verified signature
e108a7c
20:26
Upload Apollo-72B/qwen.tiktoken with huggingface_hub

Xidong

Verified signature
c2369c5
20:26
Upload Apollo-72B/model-00017-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
4ac2d83
20:17
Upload Apollo-72B/model-00021-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
4615081
20:07
Upload Apollo-72B/cache_autogptq_cuda_kernel_256.cu with huggingface_hub

Xidong

Verified signature
b3db318
20:07
Upload Apollo-72B/model-00020-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
ed7ff66
19:58
Upload Apollo-72B/model.safetensors.index.json with huggingface_hub

Xidong

Verified signature
d345ffb
19:58
Upload Apollo-72B/model-00013-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
2e8fbf1
19:49
Upload Apollo-72B/model-00016-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
6fea671
19:38
Upload Apollo-72B/model-00006-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
3805646
19:26
Upload Apollo-72B/tokenizer_config.json with huggingface_hub

Xidong

Verified signature
8942b16
19:26
Upload Apollo-72B/tokenization_qwen.py with huggingface_hub

Xidong

Verified signature
97b0290
19:26
Upload Apollo-72B/cache_autogptq_cuda_256.cpp with huggingface_hub

Xidong

Verified signature
ab61e53
19:26
Upload Apollo-72B/cpp_kernels.py with huggingface_hub

Xidong

Verified signature
ef4a7ca
19:26
Upload Apollo-72B/model-00009-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
f3f1bb7
19:08
Upload Apollo-72B/model-00002-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
fd4ffa3
18:50
Upload Apollo-72B/model-00031-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
077490c
18:41
Upload Apollo-72B/model-00028-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
382d174
18:25
Upload Apollo-72B/model-00018-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
0b65835
18:10
Upload Apollo-72B/model-00024-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
4cbe45f
17:40
Upload Apollo-72B/qwen_generation_utils.py with huggingface_hub

Xidong

Verified signature
8de2ec1
17:40
Upload Apollo-72B/config.json with huggingface_hub

Xidong

Verified signature
aaedddf
17:40
Upload Apollo-72B/model-00010-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
c118562
16:45
Upload Apollo-72B/model-00027-of-00031.safetensors with huggingface_hub

Xidong

Verified signature
c55c806

Commit History

Apr 23, 2024

Upload ./model-00005-of-00031.safetensors with huggingface_hub

Upload result.png

Upload README.md

Upload result.png

Upload README.md

Upload 3 files

Create assets/logo

Apr 22, 2024

Upload Apollo-72B/model-00029-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00003-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/generation_config.json with huggingface_hub

Upload Apollo-72B/model-00026-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00011-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00014-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00004-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00001-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00025-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00007-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00023-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00015-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00012-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00019-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00022-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00008-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00030-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/modeling_qwen.py with huggingface_hub

Upload Apollo-72B/special_tokens_map.json with huggingface_hub

Upload Apollo-72B/configuration_qwen.py with huggingface_hub

Upload Apollo-72B/qwen.tiktoken with huggingface_hub

Upload Apollo-72B/model-00017-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00021-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/cache_autogptq_cuda_kernel_256.cu with huggingface_hub

Upload Apollo-72B/model-00020-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model.safetensors.index.json with huggingface_hub

Upload Apollo-72B/model-00013-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00016-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00006-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/tokenizer_config.json with huggingface_hub

Upload Apollo-72B/tokenization_qwen.py with huggingface_hub

Upload Apollo-72B/cache_autogptq_cuda_256.cpp with huggingface_hub

Upload Apollo-72B/cpp_kernels.py with huggingface_hub

Upload Apollo-72B/model-00009-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00002-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00031-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00028-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00018-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00024-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/qwen_generation_utils.py with huggingface_hub

Upload Apollo-72B/config.json with huggingface_hub

Upload Apollo-72B/model-00010-of-00031.safetensors with huggingface_hub

Upload Apollo-72B/model-00027-of-00031.safetensors with huggingface_hub