Instructions to use FreedomIntelligence/AceGPT-13B-chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FreedomIntelligence/AceGPT-13B-chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FreedomIntelligence/AceGPT-13B-chat")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("FreedomIntelligence/AceGPT-13B-chat")
model = AutoModelForCausalLM.from_pretrained("FreedomIntelligence/AceGPT-13B-chat")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use FreedomIntelligence/AceGPT-13B-chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FreedomIntelligence/AceGPT-13B-chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/AceGPT-13B-chat",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/FreedomIntelligence/AceGPT-13B-chat

SGLang

How to use FreedomIntelligence/AceGPT-13B-chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FreedomIntelligence/AceGPT-13B-chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/AceGPT-13B-chat",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FreedomIntelligence/AceGPT-13B-chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/AceGPT-13B-chat",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use FreedomIntelligence/AceGPT-13B-chat with Docker Model Runner:
```
docker model run hf.co/FreedomIntelligence/AceGPT-13B-chat
```

jianqing666 commited on Sep 21, 2023

Commit

e964366

1 Parent(s): 90697af

Update README.md

Browse files

Files changed (1) hide show

README.md +4 -4

README.md CHANGED Viewed

@@ -5,22 +5,22 @@ language:
 ---
 # <b>AceGPT</b>
 AceGPT is a fully fine-tuned generative text model collection based on LlaMA2, particularly in the
-Arabic language domain. This is the repository for the 13B-chat pretrained model.
 ---
 ## Model Details
 We have released the AceGPT family of large language models, which is a collection of fully fine-tuned generative text models based on LlaMA2, ranging from 7B to 13B parameters. Our models include two main categories: AceGPT and AceGPT-chat. AceGPT-chat is an optimized version specifically designed for dialogue applications. It is worth mentioning that our models have demonstrated superior performance compared to all currently available open-source Arabic dialogue models in multiple benchmark tests. Furthermore, in our human evaluations, our models have shown comparable satisfaction levels to some closed-source models, such as ChatGPT, in the Arabic language.
 ## Model Developers
-We are from the School of Data Science, the Chinese University of Hong Kong, Shenzhen (CUHKSZ), and the Shenzhen Research Institute of Big Data (SRIBD).
 ## Variations
-AceGPT famils comes in a range of parameter sizes —— 7B and 13B, each size of model has a base categorie and a -chat categorie.
 ## Input
 Models input text only.
 ## Output
 Models output text only.
 ## Model Evaluation Results
-Experiments on Arabic Vicuna-80, Arabic AlpacaEval. Numbers are the average perfor-mance ratio of ChatGPT over three runs. We do not report results of raw Llama-2 models since they cannot properly generate Arabic texts.
 |                              | Arabic Vicuna-80   | Arabic AlpacaEval   |
 |------------------------------|--------------------|---------------------|
 | Phoenix Chen et al. (2023a)                | 71.92% ± 0.2%      | 65.62% ± 0.3%       |

 ---
 # <b>AceGPT</b>
 AceGPT is a fully fine-tuned generative text model collection based on LlaMA2, particularly in the
+Arabic language domain. This is the repository for the 13B-chat pre-trained model.
 ---
 ## Model Details
 We have released the AceGPT family of large language models, which is a collection of fully fine-tuned generative text models based on LlaMA2, ranging from 7B to 13B parameters. Our models include two main categories: AceGPT and AceGPT-chat. AceGPT-chat is an optimized version specifically designed for dialogue applications. It is worth mentioning that our models have demonstrated superior performance compared to all currently available open-source Arabic dialogue models in multiple benchmark tests. Furthermore, in our human evaluations, our models have shown comparable satisfaction levels to some closed-source models, such as ChatGPT, in the Arabic language.
 ## Model Developers
+We are from the School of Data Science, the Chinese University of Hong Kong, Shenzhen (CUHKSZ), the Shenzhen Research Institute of Big Data (SRIBD), and the King Abdullah University of Science and Technology (KAUST).
 ## Variations
+AceGPT families come in a range of parameter sizes —— 7B and 13B, each size of model has a base category and a -chat category.
 ## Input
 Models input text only.
 ## Output
 Models output text only.
 ## Model Evaluation Results
+Experiments on Arabic Vicuna-80, Arabic AlpacaEval. Numbers are the average performance ratio of ChatGPT over three runs. We do not report the results of raw Llama-2 models since they cannot properly generate Arabic texts.
 |                              | Arabic Vicuna-80   | Arabic AlpacaEval   |
 |------------------------------|--------------------|---------------------|
 | Phoenix Chen et al. (2023a)                | 71.92% ± 0.2%      | 65.62% ± 0.3%       |