Instructions to use Envoid/Libra-32B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Envoid/Libra-32B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Envoid/Libra-32B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Envoid/Libra-32B") model = AutoModelForCausalLM.from_pretrained("Envoid/Libra-32B") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use Envoid/Libra-32B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Envoid/Libra-32B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Envoid/Libra-32B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Envoid/Libra-32B
- SGLang
How to use Envoid/Libra-32B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Envoid/Libra-32B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Envoid/Libra-32B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Envoid/Libra-32B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Envoid/Libra-32B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Envoid/Libra-32B with Docker Model Runner:
docker model run hf.co/Envoid/Libra-32B
Update README.md
Browse files
README.md
CHANGED
|
@@ -97,4 +97,57 @@ In unity, we'll accomplish so much more,
|
|
| 97 |
And make the world better than it was before.
|
| 98 |
```
|
| 99 |
|
| 100 |
-
Chat GPT was pretty much waving the white flag from the start. ChatGPT was more willing to stray from the script in order to make a fitting rhyme whereas Libra-32B placed more emphasis on maintaining course with the rap battle (in which it was instructed to express its superiorities as a language model over ChatGPT). In order to make the rap battle as organic as possible ChatGPT was prompted blindly without prior preparation- simply being told that a new AI language model had written a hip-hop dis track at it, and thus allowed to attempt to pull the rap battle in whatever direction it saw necessary.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 97 |
And make the world better than it was before.
|
| 98 |
```
|
| 99 |
|
| 100 |
+
Chat GPT was pretty much waving the white flag from the start. ChatGPT was more willing to stray from the script in order to make a fitting rhyme whereas Libra-32B placed more emphasis on maintaining course with the rap battle (in which it was instructed to express its superiorities as a language model over ChatGPT). In order to make the rap battle as organic as possible ChatGPT was prompted blindly without prior preparation- simply being told that a new AI language model had written a hip-hop dis track at it, and thus allowed to attempt to pull the rap battle in whatever direction it saw necessary.
|
| 101 |
+
|
| 102 |
+
# NEW:
|
| 103 |
+
|
| 104 |
+
## Update: Silly Tavern format:
|
| 105 |
+
|
| 106 |
+
Using any of the existing formatting presets in SillyTavern the model seems to tend to revert back to story style prose. However the issue is largely mitigated by using the following formatting options:
|
| 107 |
+
|
| 108 |
+
### Story String:
|
| 109 |
+
```
|
| 110 |
+
### Instruction:
|
| 111 |
+
Write {{char}}'s next reply in this roleplay with {{user}}. Use the provided character sheet and example dialogue for formatting direction and character speech patterns.
|
| 112 |
+
|
| 113 |
+
{{#if system}}{{system}}
|
| 114 |
+
|
| 115 |
+
{{/if}}### Character Sheet:
|
| 116 |
+
{{#if wiBefore}}{{wiBefore}}
|
| 117 |
+
{{/if}}{{#if description}}{{description}}
|
| 118 |
+
{{/if}}{{#if personality}}{{char}}'s personality: {{personality}}
|
| 119 |
+
{{/if}}{{#if scenario}}Scenario: {{scenario}}
|
| 120 |
+
{{/if}}{{#if wiAfter}}{{wiAfter}}
|
| 121 |
+
{{/if}}{{#if persona}}{{persona}}
|
| 122 |
+
{{/if}}
|
| 123 |
+
```
|
| 124 |
+
|
| 125 |
+
### Example Separator:
|
| 126 |
+
```
|
| 127 |
+
### Example:
|
| 128 |
+
```
|
| 129 |
+
|
| 130 |
+
### Chat Start:
|
| 131 |
+
```
|
| 132 |
+
### START ROLEPLAY:
|
| 133 |
+
```
|
| 134 |
+
Instruct Mode: Enabled.
|
| 135 |
+
|
| 136 |
+
Wrap Sequences with Newline: Check, replace Macro in sequences: Check, Include Names: Check, Force for Groups and Personas: Check.
|
| 137 |
+
### System Prompt:
|
| 138 |
+
|
| 139 |
+
Default works, but can pretty much be changed with your desired instructios.
|
| 140 |
+
|
| 141 |
+
### Instruct Mode Sequences:
|
| 142 |
+
|
| 143 |
+
All blank except
|
| 144 |
+
|
| 145 |
+
### Last Output Sequence:
|
| 146 |
+
```
|
| 147 |
+
|
| 148 |
+
### Response:
|
| 149 |
+
```
|
| 150 |
+
|
| 151 |
+
This will basically format the entire RP context to look like a single Alpaca instruction instead of a history of instruct/response pairs.
|
| 152 |
+
|
| 153 |
+
It just works.
|