Update README.md
Browse files
README.md
CHANGED
|
@@ -78,6 +78,19 @@ completion = client.chat.completions.create(
|
|
| 78 |
)
|
| 79 |
```
|
| 80 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
## With Tool-use
|
| 82 |
To enable tool-use, load, tokenize, and generate as the examples above, but additionally define a set of tools and pass them to the chat template.
|
| 83 |
|
|
|
|
| 78 |
)
|
| 79 |
```
|
| 80 |
|
| 81 |
+
## Using specific checkpoints
|
| 82 |
+
To use a specific checkpoint, use the `revision` argument:
|
| 83 |
+
|
| 84 |
+
**Using transformers**
|
| 85 |
+
```
|
| 86 |
+
model = AutoModelForCausalLM.from_pretrained("llm360/k2-v2-instruct", device_map="auto", revision=stage_1_0135000)
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
**Using vLLM**
|
| 90 |
+
```
|
| 91 |
+
vllm serve LLM360/K2-V2-Instruct --tensor-parallel-size 8 --port 8000 --revision stage_1_0135000
|
| 92 |
+
```
|
| 93 |
+
|
| 94 |
## With Tool-use
|
| 95 |
To enable tool-use, load, tokenize, and generate as the examples above, but additionally define a set of tools and pass them to the chat template.
|
| 96 |
|