Text Generation
Transformers
PyTorch
TensorBoard
Chinese
llama
llama2
chinese-llama2
gogpt2-7b
text-generation-inference
Instructions to use golaxy/gogpt2-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use golaxy/gogpt2-7b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="golaxy/gogpt2-7b")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("golaxy/gogpt2-7b") model = AutoModelForCausalLM.from_pretrained("golaxy/gogpt2-7b") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use golaxy/gogpt2-7b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "golaxy/gogpt2-7b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "golaxy/gogpt2-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/golaxy/gogpt2-7b
- SGLang
How to use golaxy/gogpt2-7b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "golaxy/gogpt2-7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "golaxy/gogpt2-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "golaxy/gogpt2-7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "golaxy/gogpt2-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use golaxy/gogpt2-7b with Docker Model Runner:
docker model run hf.co/golaxy/gogpt2-7b
quincyqiang commited on
Commit ·
ee60ed4
1
Parent(s): d5e09dd
update
Browse files- README.md +2 -1
- ghostdriver.log +5 -0
README.md
CHANGED
|
@@ -148,4 +148,5 @@ print(sorted_arr)
|
|
| 148 |
|
| 149 |
模型生成的内容受模型计算、随机性和量化精度损失等因素影响,本项目不对其准确性作出保证。
|
| 150 |
|
| 151 |
-
对于模型输出的任何内容,本项目不承担任何法律责任,亦不对因使用相关资源和输出结果而可能产生的任何损失承担责任。
|
|
|
|
|
|
| 148 |
|
| 149 |
模型生成的内容受模型计算、随机性和量化精度损失等因素影响,本项目不对其准确性作出保证。
|
| 150 |
|
| 151 |
+
对于模型输出的任何内容,本项目不承担任何法律责任,亦不对因使用相关资源和输出结果而可能产生的任何损失承担责任。
|
| 152 |
+
|
ghostdriver.log
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[INFO - 2023-07-24T05:37:46.437Z] GhostDriver - Main - running on port 53956
|
| 2 |
+
[INFO - 2023-07-24T05:37:47.366Z] Session [389427e0-29e4-11ee-858a-a7c13662ba46] - page.settings - {"XSSAuditingEnabled":false,"javascriptCanCloseWindows":true,"javascriptCanOpenWindows":true,"javascriptEnabled":true,"loadImages":true,"localToRemoteUrlAccessEnabled":false,"userAgent":"Mozilla/5.0 (Unknown; Linux x86_64) AppleWebKit/538.1 (KHTML, like Gecko) PhantomJS/2.1.1 Safari/538.1","webSecurityEnabled":true}
|
| 3 |
+
[INFO - 2023-07-24T05:37:47.366Z] Session [389427e0-29e4-11ee-858a-a7c13662ba46] - page.customHeaders: - {}
|
| 4 |
+
[INFO - 2023-07-24T05:37:47.366Z] Session [389427e0-29e4-11ee-858a-a7c13662ba46] - Session.negotiatedCapabilities - {"browserName":"phantomjs","version":"2.1.1","driverName":"ghostdriver","driverVersion":"1.2.0","platform":"linux-unknown-64bit","javascriptEnabled":true,"takesScreenshot":true,"handlesAlerts":false,"databaseEnabled":false,"locationContextEnabled":false,"applicationCacheEnabled":false,"browserConnectionEnabled":false,"cssSelectorsEnabled":true,"webStorageEnabled":false,"rotatable":false,"acceptSslCerts":false,"nativeEvents":true,"proxy":{"proxyType":"direct"}}
|
| 5 |
+
[INFO - 2023-07-24T05:37:47.366Z] SessionManagerReqHand - _postNewSessionCommand - New Session Created: 389427e0-29e4-11ee-858a-a7c13662ba46
|