MiniMaxAI

/

MiniMax-M2

like 1.39k

Follow

MiniMax 2.77k

请问一下您使用的 vllm 是什么版本？使用最新的 main 分支或者 release 版本能否正常推理？

vllm 在 #29966 后变更了 rotary_dim 的语义，导致 m2 模型 rope 配置错误。我在 #30384 修复了本仓库 M2 的推理，但这会破坏早期的 fork（包括一些社区量化版本如 #30445）。最终 #30389 彻底修复了该问题。

总而言之，最新的 main 分支或者 release 版本应该不存在乱码问题。

Hi， 您也可以参考M2 vLLM recipe 安装最新的vLLM再测试下

@rogeryoungh @jeeejeee vllm版本是0.12.0

用sglang部署后，推理也是乱码...
Finish: obj=GenerateReqInput(validation_time=7.3928385972976685e-06, received_time=1765535771.5745914, received_time_perf=8186638.534562078, rid='8d55467a10f4498ba918ed5aa293a20f', http_worker_ipc=None, text=']!b[]b]system\nYou are a helpful assistant.[e[\n]b]user\n介绍一下你自己[e[\n]b]ai\n\n', input_ids=[200034, 200019, 28463, 10, 2985, 457, 258, 12473, 23413, 46, 200020, 10, 200019, 3995, 10, 112508, 60103, 200020, 10, 200019, 1361, 10, 200050, 10], input_embeds=None, image_data=None, video_data=None, audio_data=None, sampling_params={'temperature': 1.0, 'max_new_tokens': 256, 'min_new_tokens': 0, 'stop': None, 'stop_token_ids': None, 'stop_regex': None, 'top_p': 0.95, 'top_k': 40, 'min_p': 0.0, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'repetition_penalty': 1.0, 'regex': None, 'ebnf': None, 'n': 1, 'no_stop_trim': False, 'ignore_eos': False, 'skip_special_tokens': True, 'logit_bias': None, 'custom_params': None}, return_logprob=False, logprob_start_len=-1, top_logprobs_num=0, token_ids_logprob=None, return_text_in_logprobs=True, stream=True, log_metrics=True, return_hidden_states=False, modalities=[], session_params=None, lora_path=None, lora_id=None, custom_logit_processor=None, bootstrap_host=None, bootstrap_port=None, bootstrap_room=None, bootstrap_pair_key=None, decode_tp_size=None, reasoning=True, data_parallel_rank=None, background=False, conversation_id=None, priority=None, extra_key=None, no_logs=False, custom_labels=None, return_bytes=False, return_entropy=False), out={'text': ' \n Sh,,-H,-H\n\n+\t{\",-\tAh\",", \n\t,9::","3Hq�1h�Sw5Tw,q5Ta9,Ht9t1,Sh\t�9U.\tY9U,\"3H\t--H\x9HSh6E9UT8Sh4U\t0�4�5,Ts4H1U9T,T0S1t6?s4p0T0\t1\u1 1 0 4,2,0�1,\\r\n2,1k3,3,\ 7,7,\5c',',,c2?1w5 1 5 3,2,3?,1�0?0.1,1,t4,3,3?7.h4,c5,s3,m1,56?1,s1,1.w1,7,1.s5.s 150,9,4,s,2.s3,7?5.s3,l0,7,7 --9.s3,q9?,7,s1.s3?7 2,5,23 7\t5.s3 ,8,l8,7', 'output_ids': [1056, 2101, 44, 44, 9012, 44, 9012, 367, 43, 9, 2169, 1164, 45, 9, 23247, 61779, 1164, 1056, 9, 44, 57, 1125, 11588, 51, 72, 113, 16474, 49, 104, 16474, 15680, 53, 42183, 59424, 53, 58467, 57, 83876, 116, 57, 116, 49, 49773, 104, 9, 16474, 57, 85, 46, 9, 89, 57, 85, 97657, 51, 72, 9, 420, 72, 10024, 57, 72, 3585, 54, 69, 57, 85, 84, 56, 3585, 52, 85, 9, 48, 16474, 52, 16474, 53, 44, 44732, 52, 72, 49, 85, 57, 84, 44, 84, 48, 83, 49, 116, 54, 72188, 52, 112, 48, 84, 48, 9, 49, 13137, 49, 32, 49, 32, 48, 32, 52, 44, 50, 44, 48, 16474, 49, 44, 92, 2535, 50, 44, 49, 107, 51, 44, 51, 44, 92, 32, 55, 44, 55, 44, 92, 53, 99, 1635, 1635, 44, 99, 50, 63, 49, 119, 53, 32, 49, 32, 53, 32, 51, 44, 50, 44, 51, 35418, 49, 16474, 48, 63, 48, 46, 49, 44, 49, 25768, 52, 44, 51, 44, 51, 63, 55, 4031, 52, 40200, 53, 33698, 51, 36837, 49, 44, 53, 54, 63, 49, 33698, 49, 44, 49, 5586, 49, 44, 55, 44, 49, 1411, 53, 1411, 32, 49, 53, 48, 44, 57, 44, 52, 33698, 44, 50, 1411, 51, 44, 55, 63, 53, 1411, 51, 58284, 48, 44, 55, 44, 55, 2918, 57, 1411, 51, 59424, 57, 35418, 55, 33698, 49, 1411, 51, 63, 55, 32, 50, 44, 53, 44, 50, 51, 32, 55, 9, 53, 1411, 51, 2188, 56, 58284, 56, 44, 55], 'meta_info': {'id': '8d55467a10f4498ba918ed5aa293a20f', 'finish_reason': {'type': 'length', 'length': 256}, 'prompt_tokens': 24, 'weight_version': 'default', 'total_retractions': 0, 'completion_tokens': 256, 'cached_tokens': 23, 'e2e_latency': 26.134023666381836}}