Instructions to use platzi/chivoom with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use platzi/chivoom with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="platzi/chivoom")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("platzi/chivoom", dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use platzi/chivoom with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "platzi/chivoom" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "platzi/chivoom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/platzi/chivoom
- SGLang
How to use platzi/chivoom with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "platzi/chivoom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "platzi/chivoom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "platzi/chivoom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "platzi/chivoom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use platzi/chivoom with Docker Model Runner:
docker model run hf.co/platzi/chivoom
Update README.md
Browse files
README.md
CHANGED
|
@@ -71,17 +71,17 @@ model.eval()
|
|
| 71 |
# Based on the inference code by `tloen/alpaca-lora`
|
| 72 |
def generate_prompt(instruction, input=None):
|
| 73 |
if input:
|
| 74 |
-
return f"""
|
| 75 |
-
###
|
| 76 |
{instruction}
|
| 77 |
-
###
|
| 78 |
{input}
|
| 79 |
-
###
|
| 80 |
else:
|
| 81 |
-
return f"""
|
| 82 |
-
###
|
| 83 |
{instruction}
|
| 84 |
-
###
|
| 85 |
|
| 86 |
def generate(
|
| 87 |
instruction,
|
|
@@ -112,10 +112,10 @@ def generate(
|
|
| 112 |
)
|
| 113 |
s = generation_output.sequences[0]
|
| 114 |
output = tokenizer.decode(s)
|
| 115 |
-
return output.split("### Response:")[1]
|
| 116 |
|
| 117 |
instruction = "驴Qu茅 es un chivo?"
|
| 118 |
|
| 119 |
-
print("
|
| 120 |
-
print("
|
| 121 |
```
|
|
|
|
| 71 |
# Based on the inference code by `tloen/alpaca-lora`
|
| 72 |
def generate_prompt(instruction, input=None):
|
| 73 |
if input:
|
| 74 |
+
return f"""A continuaci贸n se muestra una instrucci贸n que describe una tarea, emparejada con una entrada que proporciona m谩s contexto. Escribe una respuesta que complete adecuadamente la petici贸n.
|
| 75 |
+
### Instrucci贸n:
|
| 76 |
{instruction}
|
| 77 |
+
### Entrada:
|
| 78 |
{input}
|
| 79 |
+
### Respuesta:"""
|
| 80 |
else:
|
| 81 |
+
return f"""A continuaci贸n se muestra una instrucci贸n que describe una tarea. Escribe una respuesta que complete adecuadamente la petici贸n.
|
| 82 |
+
### Instrucci贸n:
|
| 83 |
{instruction}
|
| 84 |
+
### Respuesta:"""
|
| 85 |
|
| 86 |
def generate(
|
| 87 |
instruction,
|
|
|
|
| 112 |
)
|
| 113 |
s = generation_output.sequences[0]
|
| 114 |
output = tokenizer.decode(s)
|
| 115 |
+
return output.split("### Response:")[1]
|
| 116 |
|
| 117 |
instruction = "驴Qu茅 es un chivo?"
|
| 118 |
|
| 119 |
+
print("Instrucci贸n:", instruction)
|
| 120 |
+
print("Respuesta:", generate(instruction))
|
| 121 |
```
|