Text Generation
Transformers
Safetensors
English
llama
Merge
mergekit
nvidia/Llama3-ChatQA-1.5-8B
refuelai/Llama-3-Refueled
text-generation-inference
Instructions to use saucam/PowerBot-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use saucam/PowerBot-8B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="saucam/PowerBot-8B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("saucam/PowerBot-8B") model = AutoModelForCausalLM.from_pretrained("saucam/PowerBot-8B") - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use saucam/PowerBot-8B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "saucam/PowerBot-8B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "saucam/PowerBot-8B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/saucam/PowerBot-8B
- SGLang
How to use saucam/PowerBot-8B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "saucam/PowerBot-8B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "saucam/PowerBot-8B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "saucam/PowerBot-8B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "saucam/PowerBot-8B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use saucam/PowerBot-8B with Docker Model Runner:
docker model run hf.co/saucam/PowerBot-8B
Update README.md
Browse files
README.md
CHANGED
|
@@ -62,4 +62,14 @@ pipeline = transformers.pipeline(
|
|
| 62 |
|
| 63 |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
| 64 |
print(outputs[0]["generated_text"])
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
```
|
|
|
|
| 62 |
|
| 63 |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
| 64 |
print(outputs[0]["generated_text"])
|
| 65 |
+
```
|
| 66 |
+
|
| 67 |
+
```
|
| 68 |
+
Loading checkpoint shards: 100%|βββββββββββββββββββββββββββββββββββββββββββββββββββ| 2/2 [00:07<00:00, 3.75s/it]
|
| 69 |
+
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
|
| 70 |
+
Setting `pad_token_id` to `eos_token_id`:128001 for open-end generation.
|
| 71 |
+
<|im_start|>user
|
| 72 |
+
What is a large language model?<|im_end|>
|
| 73 |
+
<|im_start|>assistant
|
| 74 |
+
A large language model (LLM) is a deep neural network that is trained to predict the next word in a sequence of text. LLMs are typically trained on large amounts of text data and can be used for a variety of tasks such as language translation, text completion, and question answering. They are often used to generate human-like text and are becoming increasingly popular in natural language processing applications. The LLM uses a transformer architecture, which consists of multiple layers of neural networks that are trained to process and understand the relationships between words in a sentence. The transformer architecture is designed to handle long sequences of text and is capable of capturing the context of a word within a sentence. This allows the LLM to generate coherent and grammatically correct text that is similar to human writing. LLMs are typically trained on a large corpus of text data and can be fine-tuned for specific tasks by retraining on smaller datasets that are relevant to the task at hand. This allows the LLM to adapt to the specific requirements of a particular application and improve its performance. The LLM can be used to generate text in a variety of formats, including natural language, code, and even mathematical expressions. It can also be used to translate text from one language to another, generate summaries of
|
| 75 |
```
|