Instructions to use distributed/optimized-gpt2-1b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use distributed/optimized-gpt2-1b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="distributed/optimized-gpt2-1b", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("distributed/optimized-gpt2-1b", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use distributed/optimized-gpt2-1b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "distributed/optimized-gpt2-1b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "distributed/optimized-gpt2-1b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/distributed/optimized-gpt2-1b
- SGLang
How to use distributed/optimized-gpt2-1b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "distributed/optimized-gpt2-1b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "distributed/optimized-gpt2-1b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "distributed/optimized-gpt2-1b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "distributed/optimized-gpt2-1b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use distributed/optimized-gpt2-1b with Docker Model Runner:
docker model run hf.co/distributed/optimized-gpt2-1b
Commit History
-
Run 4. Outer Step 103. Inner Step 0. Peers 72.
Verified signature6fbb3aa -
Run 4. Outer Step 102. Inner Step 0. Peers 71.
Verified signature9c23506 -
Run 4. Outer Step 101. Inner Step 0. Peers 68.
Verified signature07b13d4
-
Run 4. Outer Step 100. Inner Step 0. Peers 67.
Verified signature3faf662 -
Run 4. Outer Step 99. Inner Step 0. Peers 67.
Verified signature9225d98 -
Run 4. Outer Step 98. Inner Step 0. Peers 63.
Verified signature9d7b0c6 -
Run 4. Outer Step 97. Inner Step 0. Peers 54.
Verified signature0ff8b42 -
Run 4. Outer Step 96. Inner Step 0. Peers 50.
Verified signature8c3907a -
Run 4. Outer Step 95. Inner Step 0. Peers 39.
Verified signaturea363dee
-
Run 4. Outer Step 94. Inner Step 0. Peers 21.
Verified signaturec9a53b3 -
Run 4. Outer Step 93. Inner Step 0. Peers 73.
Verified signaturea2c62b8 -
Run 4. Outer Step 92. Inner Step 0. Peers 83.
Verified signature0156c66 -
Run 4. Outer Step 91. Inner Step 0. Peers 81.
Verified signature30df776 -
Run 4. Outer Step 90. Inner Step 0. Peers 86.
Verified signature2c81937
-
Run 4. Outer Step 89. Inner Step 0. Peers 83.
Verified signature13d116f -
Run 4. Outer Step 88. Inner Step 0. Peers 83.
Verified signaturef75ad78 -
Run 4. Outer Step 87. Inner Step 0. Peers 83.
Verified signaturec5f6854 -
Run 4. Outer Step 86. Inner Step 0. Peers 83.
Verified signature03f1160 -
Run 4. Outer Step 85. Inner Step 0. Peers 81.
Verified signature2efe7c4
-
Run 4. Outer Step 84. Inner Step 0. Peers 82.
Verified signaturea312830 -
Run 4. Outer Step 83. Inner Step 0. Peers 32.
Verified signature9506478 -
Run 4. Outer Step 82. Inner Step 0. Peers 100.
Verified signaturefb5221f -
Run 4. Outer Step 81. Inner Step 0. Peers 98.
Verified signaturee6fa6eb -
Run 4. Outer Step 80. Inner Step 0. Peers 97.
Verified signature1759fc8
-
Run 4. Outer Step 79. Inner Step 0. Peers 96.
Verified signature79df68d -
Run 4. Outer Step 78. Inner Step 0. Peers 35.
Verified signature75f2f74 -
Run 4. Outer Step 77. Inner Step 0. Peers 96.
Verified signature498c45d -
Run 4. Outer Step 76. Inner Step 0. Peers 20.
Verified signature82089d4 -
Run 4. Outer Step 75. Inner Step 0. Peers 21.
Verified signature448858b -
Run 4. Outer Step 74. Inner Step 0. Peers 97.
Verified signatured667c58
-
Run 4. Outer Step 73. Inner Step 0. Peers 71.
Verified signature0b45f8d -
Run 4. Outer Step 72. Inner Step 0. Peers 97.
Verified signature4318020 -
Run 4. Outer Step 71. Inner Step 0. Peers 91.
Verified signaturee4b68af -
Run 4. Outer Step 70. Inner Step 0. Peers 97.
Verified signature68acd2c -
Run 4. Outer Step 69. Inner Step 0. Peers 97.
Verified signatureff73dbb -
Run 4. Outer Step 68. Inner Step 0. Peers 95.
Verified signatured65bd16
-
Run 4. Outer Step 67. Inner Step 0. Peers 97.
Verified signature68c7c99 -
Run 4. Outer Step 66. Inner Step 0. Peers 98.
Verified signaturee592eae -
Run 4. Outer Step 65. Inner Step 0. Peers 97.
Verified signature92b6fe6 -
Run 4. Outer Step 64. Inner Step 0. Peers 71.
Verified signaturefcc37f6
-
Run 4. Outer Step 63. Inner Step 0. Peers 68.
Verified signature43acc40 -
Run 4. Outer Step 62. Inner Step 0. Peers 95.
Verified signature5d56280 -
Run 4. Outer Step 61. Inner Step 0. Peers 97.
Verified signature0751425 -
Run 4. Outer Step 60. Inner Step 0. Peers 92.
Verified signature2722729
-
Run 4. Outer Step 59. Inner Step 0. Peers 82.
Verified signature69c2270 -
Run 4. Outer Step 58. Inner Step 0. Peers 82.
Verified signatured6c180b -
Run 4. Outer Step 57. Inner Step 0. Peers 95.
Verified signature582657b -
Run 4. Outer Step 56. Inner Step 0. Peers 90.
Verified signature8728ea3 -
Run 4. Outer Step 55. Inner Step 0. Peers 92.
Verified signaturec365ea3 -
Run 4. Outer Step 54. Inner Step 0. Peers 94.
Verified signature4eb11b2