richardmfan commited on
Commit
56dd218
·
verified ·
1 Parent(s): 8727347

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -0
README.md CHANGED
@@ -78,6 +78,19 @@ completion = client.chat.completions.create(
78
  )
79
  ```
80
 
 
 
 
 
 
 
 
 
 
 
 
 
 
81
  ## With Tool-use
82
  To enable tool-use, load, tokenize, and generate as the examples above, but additionally define a set of tools and pass them to the chat template.
83
 
 
78
  )
79
  ```
80
 
81
+ ## Using specific checkpoints
82
+ To use a specific checkpoint, use the `revision` argument:
83
+
84
+ **Using transformers**
85
+ ```
86
+ model = AutoModelForCausalLM.from_pretrained("llm360/k2-v2-instruct", device_map="auto", revision=stage_1_0135000)
87
+ ```
88
+
89
+ **Using vLLM**
90
+ ```
91
+ vllm serve LLM360/K2-V2-Instruct --tensor-parallel-size 8 --port 8000 --revision stage_1_0135000
92
+ ```
93
+
94
  ## With Tool-use
95
  To enable tool-use, load, tokenize, and generate as the examples above, but additionally define a set of tools and pass them to the chat template.
96