RabotniKuma commited on
Commit
1c38a1a
·
verified ·
1 Parent(s): 820f4fb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +22 -0
README.md CHANGED
@@ -32,3 +32,25 @@ Technical details can be found in [Kaggle Discussion](https://www.kaggle.com/com
32
  # Dataset
33
  - [Our first stage SFT dataset](https://huggingface.co/datasets/RabotniKuma/Fast-Math-R1-SFT)
34
  - [Our second stage GRPO dataset](https://huggingface.co/datasets/RabotniKuma/Fast-Math-R1-GRPO)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
  # Dataset
33
  - [Our first stage SFT dataset](https://huggingface.co/datasets/RabotniKuma/Fast-Math-R1-SFT)
34
  - [Our second stage GRPO dataset](https://huggingface.co/datasets/RabotniKuma/Fast-Math-R1-GRPO)
35
+
36
+ # Inference
37
+ ## vLLM
38
+ ```python
39
+ from vllm import LLM, SamplingParams
40
+
41
+
42
+ vllm_engine = LLM(
43
+ model='RabotniKuma/Fast-Math-R1-14B',
44
+ max_model_len=8192,
45
+ gpu_memory_utilization=0.9,
46
+ trust_remote_code=True,
47
+ )
48
+ sampling_params = SamplingParams(
49
+ temperature=1.0,
50
+ top_p=0.90,
51
+ min_p=0.05,
52
+ max_tokens=8192,
53
+ stop='</think>', # Important: early stop at </think> to save output tokens
54
+ )
55
+ vllm_engine.generate('1+1=', sampling_params=sampling_params)
56
+ ```