add vllm inference example

README.md

@@ -18,11 +18,11 @@ MiniCPM3-4B is the 3rd generation of MiniCPM series. The overall performance of
 
 Compared to MiniCPM1.0/MiniCPM2.0, MiniCPM3-4B has a more powerful and versatile skill set to enable more general usage. MiniCPM3-4B supports function call, along with code interpreter. Please refer to []() for usage guidelines.
 
-MiniCPM3-4B has a 32k context window. Equipped with LLMxMapreduce, MiniCPM3-4B can handle infinite contexts theoretically, without requiring huge amount of memory.
+MiniCPM3-4B has a 32k context window. Equipped with LLMxMapReduce, MiniCPM3-4B can handle infinite contexts theoretically, without requiring huge amount of memory.
 
 ## Usage
+### Inference with Transformers
 ```python
-
 from transformers import AutoModelForCausalLM, AutoTokenizer
 import torch
 
@@ -42,7 +42,7 @@ model_outputs = model.generate(
     max_new_tokens=1024,
     top_p=0.7,
     temperature=0.7,
-    repetition_penalty=1.02    
+    repetition_penalty=1.02
 )
 
 output_token_ids = [
@@ -53,6 +53,29 @@ responses = tokenizer.batch_decode(output_token_ids, skip_special_tokens=True)[0
 print(responses)
 ```
 
+### Inference with [vLLM](https://github.com/vllm-project/vllm)
+```python
+from transformers import AutoTokenizer
+from vllm import LLM, SamplingParams
+
+model_name = "openbmb/MiniCPM3-4B"
+prompt = [{"role": "user", "content": "推荐5个北京的景点。"}]
+
+tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
+input_text = tokenizer.apply_chat_template(prompt, tokenize=False, add_generation_prompt=True)
+
+llm = LLM(
+    model=model_name,
+    trust_remote_code=True,
+    tensor_parallel_size=1
+)
+sampling_params = SamplingParams(top_p=0.7, temperature=0.7, max_tokens=1024, repetition_penalty=1.02)
+
+outputs = llm.generate(prompts=input_text, sampling_params=sampling_params)
+
+print(outputs[0].outputs[0].text)
+```
+
 ## Evaluation Results
 
 <table>