更新转换脚本和文档（claude写的，感觉也不是特别好）

README.md

@@ -78,7 +78,7 @@ time_total_sec: 31.765
 
 ## 模型转换
 
-### TODO
+https://huggingface.co/happyme531/Qwen3-ASR-1.7B-RKLLM/blob/main/convert/README.md
 
 ## 已知问题
 
@@ -157,7 +157,7 @@ time_total_sec: 31.765
 
 ### Model Conversion
 
-#### TODO
+https://huggingface.co/happyme531/Qwen3-ASR-1.7B-RKLLM/blob/main/convert/README.md
 
 ### Known Issues
 

convert/README.md

@@ -0,0 +1,171 @@
+# Qwen3-ASR-1.7B → RK3588 模型转换
+
+### (English README see below)
+
+把 [Qwen/Qwen3-ASR-1.7B](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) 转成 RK3588 上可运行的 RKNN + RKLLM。
+
+- 音频编码器（`thinker.audio_tower`）→ ONNX → **RKNN**
+- 文本 LLM（`thinker.model + thinker.lm_head`）→ 标准 `Qwen3ForCausalLM` → **RKLLM**
+- 全程 **FP16，不量化**
+
+## 目录
+
+```
+convert/
+├── audio_encoder/
+│   ├── common.py                       共享工具
+│   ├── export_audio_encoder_onnx.py    PyTorch → ONNX
+│   ├── export_audio_encoder_rknn.py    ONNX → RKNN
+│   └── onnx_run_audio_encoder.py       ONNX 对齐校验（可选）
+└── llm/
+    ├── extract_qwen3_text_model.py     抽取标准 Qwen3 文本权重
+    └── export_rkllm_direct.py          HF → RKLLM
+```
+
+## 准备
+
+把原始模型放到工作目录，并把官方 [QwenLM/Qwen3-ASR](https://github.com/QwenLM/Qwen3-ASR) 仓库克隆到同级目录（`audio_encoder/common.py` 需要从里面 import `modeling_qwen3_asr`）：
+
+```bash
+huggingface-cli download Qwen/Qwen3-ASR-1.7B --local-dir .
+git clone https://github.com/QwenLM/Qwen3-ASR.git
+```
+
+主机依赖：`torch transformers safetensors numpy scipy soundfile onnx onnxruntime`，以及 [`rknn-toolkit2`](https://github.com/airockchip/rknn-toolkit2) 和 [`rkllm-toolkit`](https://github.com/airockchip/rknn-llm)。
+
+## 1. LLM → RKLLM
+
+先抽出干净的 Qwen3 文本权重（直接喂原模型会因为残留的 `mrope/vision_config` 字段被 RKLLM 误判成视觉模型）：
+
+```bash
+python convert/llm/extract_qwen3_text_model.py \
+  --model-path . \
+  --output-dir ./qwen3_text_hf
+```
+
+然后转 RKLLM：
+
+```bash
+python convert/llm/export_rkllm_direct.py \
+  --model-path ./qwen3_text_hf \
+  --target-platform rk3588 --num-npu-core 3 \
+  --dtype float16 --max-context 4096 \
+  --savepath ./rknn/language_model.rkllm
+```
+
+## 2. 音频编码器 → RKNN
+
+音频塔被包成「100 mel 帧 / chunk」的静态模型（这是原模型本身的处理粒度），长音频在运行时分块跑再拼回去。
+
+```bash
+# PyTorch → ONNX
+python convert/audio_encoder/export_audio_encoder_onnx.py \
+  --model-path . --savepath ./onnx/qwen3_asr_audio_chunk100.onnx
+
+# (可选) 对齐校验，正常 max_abs_diff ≈ 1e-7
+python convert/audio_encoder/onnx_run_audio_encoder.py \
+  --model-path . --onnx-path ./onnx/qwen3_asr_audio_chunk100.onnx \
+  --audio-path asr_example_zh.wav --compare-torch
+
+# ONNX → RKNN
+python convert/audio_encoder/export_audio_encoder_rknn.py \
+  --onnx-path ./onnx/qwen3_asr_audio_chunk100.onnx \
+  --target-platform rk3588 --savepath ./rknn/audio_encoder.rknn
+```
+
+## 3. 产物
+
+```
+rknn/
+├── audio_encoder.rknn
+└── language_model.rkllm
+```
+
+直接对接仓库根目录的 `run_qwen3_asr_e2e.py`。
+
+---
+
+# Qwen3-ASR-1.7B → RK3588 Model Conversion
+
+Convert [Qwen/Qwen3-ASR-1.7B](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) to RKNN + RKLLM for RK3588.
+
+- Audio encoder (`thinker.audio_tower`) → ONNX → **RKNN**
+- Text LLM (`thinker.model + thinker.lm_head`) → standard `Qwen3ForCausalLM` → **RKLLM**
+- **FP16 throughout, no quantization**
+
+## Layout
+
+```
+convert/
+├── audio_encoder/
+│   ├── common.py                       shared helpers
+│   ├── export_audio_encoder_onnx.py    PyTorch → ONNX
+│   ├── export_audio_encoder_rknn.py    ONNX → RKNN
+│   └── onnx_run_audio_encoder.py       ONNX parity check (optional)
+└── llm/
+    ├── extract_qwen3_text_model.py     extract standard Qwen3 text weights
+    └── export_rkllm_direct.py          HF → RKLLM
+```
+
+## Setup
+
+Place the original model in your working directory and clone the official [QwenLM/Qwen3-ASR](https://github.com/QwenLM/Qwen3-ASR) repo as a sibling (`audio_encoder/common.py` imports `modeling_qwen3_asr` from it):
+
+```bash
+huggingface-cli download Qwen/Qwen3-ASR-1.7B --local-dir .
+git clone https://github.com/QwenLM/Qwen3-ASR.git
+```
+
+Host dependencies: `torch transformers safetensors numpy scipy soundfile onnx onnxruntime`, plus [`rknn-toolkit2`](https://github.com/airockchip/rknn-toolkit2) and [`rkllm-toolkit`](https://github.com/airockchip/rknn-llm).
+
+## 1. LLM → RKLLM
+
+First extract clean Qwen3 text weights (feeding the original model directly trips RKLLM into thinking it's a vision model because of leftover `mrope/vision_config` fields):
+
+```bash
+python convert/llm/extract_qwen3_text_model.py \
+  --model-path . \
+  --output-dir ./qwen3_text_hf
+```
+
+Then convert to RKLLM:
+
+```bash
+python convert/llm/export_rkllm_direct.py \
+  --model-path ./qwen3_text_hf \
+  --target-platform rk3588 --num-npu-core 3 \
+  --dtype float16 --max-context 4096 \
+  --savepath ./rknn/language_model.rkllm
+```
+
+## 2. Audio encoder → RKNN
+
+The audio tower is wrapped as a static "100 mel frames / chunk" model (this matches the model's native processing granularity); long audio is split, run chunk-by-chunk and concatenated at runtime.
+
+```bash
+# PyTorch → ONNX
+python convert/audio_encoder/export_audio_encoder_onnx.py \
+  --model-path . --savepath ./onnx/qwen3_asr_audio_chunk100.onnx
+
+# (optional) parity check, expect max_abs_diff ≈ 1e-7
+python convert/audio_encoder/onnx_run_audio_encoder.py \
+  --model-path . --onnx-path ./onnx/qwen3_asr_audio_chunk100.onnx \
+  --audio-path asr_example_zh.wav --compare-torch
+
+# ONNX → RKNN
+python convert/audio_encoder/export_audio_encoder_rknn.py \
+  --onnx-path ./onnx/qwen3_asr_audio_chunk100.onnx \
+  --target-platform rk3588 --savepath ./rknn/audio_encoder.rknn
+```
+
+
+## 3. Artifacts
+
+```
+rknn/
+├── audio_encoder.rknn
+└── language_model.rkllm
+```
+
+These plug directly into `run_qwen3_asr_e2e.py` at the repo root.
+

convert/audio_encoder/common.py

@@ -0,0 +1,199 @@
+import math
+import sys
+from pathlib import Path
+
+import numpy as np
+import soundfile as sf
+import torch
+from scipy.signal import resample_poly
+from transformers import WhisperFeatureExtractor
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+QWEN_ASR_CORE = REPO_ROOT / "Qwen3-ASR" / "qwen_asr" / "core"
+if str(QWEN_ASR_CORE) not in sys.path:
+    sys.path.insert(0, str(QWEN_ASR_CORE))
+
+from transformers_backend.modeling_qwen3_asr import (  # noqa: E402
+    Qwen3ASRForConditionalGeneration,
+)
+
+
+TORCH_DTYPES = {
+    "float16": torch.float16,
+    "bfloat16": torch.bfloat16,
+    "float32": torch.float32,
+}
+
+
+def get_torch_dtype(dtype: str) -> torch.dtype:
+    if dtype not in TORCH_DTYPES:
+        raise ValueError(f"Unsupported dtype: {dtype}")
+    return TORCH_DTYPES[dtype]
+
+
+def load_waveform(audio_path: str, target_sr: int = 16000) -> np.ndarray:
+    audio, sr = sf.read(audio_path, dtype="float32", always_2d=False)
+    audio = np.asarray(audio, dtype=np.float32)
+    if audio.ndim == 2:
+        audio = audio.mean(axis=-1)
+    if sr != target_sr:
+        divisor = math.gcd(int(sr), int(target_sr))
+        up = int(target_sr // divisor)
+        down = int(sr // divisor)
+        audio = resample_poly(audio, up=up, down=down).astype(np.float32)
+    return audio
+
+
+def configure_feature_extractor_for_audio(feature_extractor: WhisperFeatureExtractor, waveform: np.ndarray) -> None:
+    required_seconds = max(1, math.ceil(waveform.shape[0] / float(feature_extractor.sampling_rate)))
+    if required_seconds <= feature_extractor.chunk_length:
+        return
+    feature_extractor.chunk_length = required_seconds
+    feature_extractor.n_samples = int(required_seconds * feature_extractor.sampling_rate)
+    feature_extractor.nb_max_frames = feature_extractor.n_samples // feature_extractor.hop_length
+
+
+def extract_mel_features(model_path: str, audio_path: str) -> tuple[np.ndarray, int]:
+    feature_extractor = WhisperFeatureExtractor.from_pretrained(model_path)
+    waveform = load_waveform(audio_path)
+    configure_feature_extractor_for_audio(feature_extractor, waveform)
+    outputs = feature_extractor(
+        waveform,
+        sampling_rate=16000,
+        return_attention_mask=True,
+        return_tensors="np",
+    )
+    input_features = outputs["input_features"][0].astype(np.float32)
+    feature_len = int(outputs["attention_mask"][0].sum())
+    return input_features, feature_len
+
+
+def split_mel_features(input_features: np.ndarray, feature_len: int, chunk_frames: int) -> list[tuple[np.ndarray, int]]:
+    chunks = []
+    start = 0
+    while start < feature_len:
+        cur_len = min(chunk_frames, feature_len - start)
+        chunk = np.zeros((input_features.shape[0], chunk_frames), dtype=np.float32)
+        chunk[:, :cur_len] = input_features[:, start : start + cur_len]
+        chunks.append((chunk, cur_len))
+        start += cur_len
+    return chunks
+
+
+def load_audio_encoder(model_path: str, dtype: str = "float32", device: str = "cpu") -> torch.nn.Module:
+    model = Qwen3ASRForConditionalGeneration.from_pretrained(
+        model_path,
+        dtype=get_torch_dtype(dtype),
+    )
+    model = model.to(device)
+    model.eval()
+    tower = model.thinker.audio_tower
+    tower.config._attn_implementation = "eager"
+    for layer in tower.layers:
+        layer.self_attn.config._attn_implementation = "eager"
+    tower.eval()
+    return tower
+
+
+def get_chunk_output_length(length: torch.Tensor) -> torch.Tensor:
+    length = length.to(torch.int64)
+    length = torch.div(length + 1, 2, rounding_mode="floor")
+    length = torch.div(length + 1, 2, rounding_mode="floor")
+    length = torch.div(length + 1, 2, rounding_mode="floor")
+    return length
+
+
+def get_chunk_output_length_value(length: int) -> int:
+    value = int(length)
+    value = (value + 1) // 2
+    value = (value + 1) // 2
+    value = (value + 1) // 2
+    return value
+
+
+class StaticChunkAudioEncoder(torch.nn.Module):
+    def __init__(self, tower: torch.nn.Module, chunk_frames: int = 100):
+        super().__init__()
+        self.tower = tower
+        self.chunk_frames = int(chunk_frames)
+        self.max_aftercnn_len = int(get_chunk_output_length(torch.tensor(self.chunk_frames)).item())
+
+    def forward(self, input_features: torch.Tensor, feature_len: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
+        batch_size = input_features.shape[0]
+        outputs = []
+        valid_lens = []
+
+        for batch_idx in range(batch_size):
+            current_feature_len = feature_len.reshape(-1)[batch_idx].to(torch.int64)
+            padded = input_features[batch_idx, :, : self.chunk_frames].unsqueeze(0).unsqueeze(0)
+
+            padded_embed = torch.nn.functional.gelu(self.tower.conv2d1(padded))
+            padded_embed = torch.nn.functional.gelu(self.tower.conv2d2(padded_embed))
+            padded_embed = torch.nn.functional.gelu(self.tower.conv2d3(padded_embed))
+
+            batch, channels, freq, time_steps = padded_embed.size()
+            padded_embed = self.tower.conv_out(
+                padded_embed.permute(0, 3, 1, 2).contiguous().view(batch, time_steps, channels * freq)
+            )
+            positional_embedding = self.tower.positional_embedding.positional_embedding[:time_steps, :]
+            padded_embed = padded_embed + positional_embedding.unsqueeze(0).to(padded_embed.dtype)
+            hidden_states = padded_embed.squeeze(0)
+
+            valid_len = get_chunk_output_length(current_feature_len.unsqueeze(0))[0].to(torch.int32)
+            valid_positions = torch.arange(time_steps, device=hidden_states.device, dtype=torch.int32) < valid_len
+            allowed = valid_positions[:, None] & valid_positions[None, :]
+
+            zeros = torch.zeros((1, 1, time_steps, time_steps), dtype=hidden_states.dtype, device=hidden_states.device)
+            minus_inf = torch.full(
+                (1, 1, time_steps, time_steps),
+                torch.finfo(hidden_states.dtype).min,
+                dtype=hidden_states.dtype,
+                device=hidden_states.device,
+            )
+            attention_mask = torch.where(allowed.unsqueeze(0).unsqueeze(0), zeros, minus_inf)
+            cu_seqlens = torch.stack(
+                (
+                    torch.zeros((), dtype=torch.int32, device=hidden_states.device),
+                    valid_len,
+                )
+            )
+
+            for encoder_layer in self.tower.layers:
+                hidden_states = encoder_layer(
+                    hidden_states,
+                    cu_seqlens=cu_seqlens,
+                    attention_mask=attention_mask,
+                )[0]
+
+            hidden_states = self.tower.ln_post(hidden_states)
+            hidden_states = self.tower.proj1(hidden_states)
+            hidden_states = self.tower.act(hidden_states)
+            hidden_states = self.tower.proj2(hidden_states)
+
+            outputs.append(hidden_states.unsqueeze(0))
+            valid_lens.append(valid_len)
+
+        return torch.cat(outputs, dim=0), torch.stack(valid_lens, dim=0)
+
+
+def run_chunked_torch(
+    model_path: str,
+    input_features: np.ndarray,
+    feature_len: int,
+    chunk_frames: int = 100,
+    dtype: str = "float32",
+    device: str = "cpu",
+) -> np.ndarray:
+    tower = load_audio_encoder(model_path=model_path, dtype=dtype, device=device)
+    wrapper = StaticChunkAudioEncoder(tower=tower, chunk_frames=chunk_frames).to(device).eval()
+
+    outputs = []
+    for chunk, chunk_len in split_mel_features(input_features, feature_len, chunk_frames):
+        chunk_tensor = torch.from_numpy(chunk).unsqueeze(0).to(device=device, dtype=get_torch_dtype(dtype))
+        chunk_len_tensor = torch.tensor([chunk_len], dtype=torch.int32, device=device)
+        with torch.no_grad():
+            features, valid_len = wrapper(chunk_tensor, chunk_len_tensor)
+        outputs.append(features[0, : valid_len[0].item()].detach().cpu().numpy())
+
+    return np.concatenate(outputs, axis=0)

convert/audio_encoder/export_audio_encoder_onnx.py

@@ -0,0 +1,84 @@
+import argparse
+from pathlib import Path
+
+import onnxruntime as ort
+import torch
+
+from common import StaticChunkAudioEncoder, get_torch_dtype, load_audio_encoder
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description="Export Qwen3-ASR audio encoder chunk model to ONNX.")
+    parser.add_argument("--model-path", type=str, default=".", help="Path to Qwen3-ASR model directory.")
+    parser.add_argument(
+        "--savepath",
+        type=str,
+        default="rknn_deploy/audio_encoder/onnx/qwen3_asr_audio_chunk100.onnx",
+        help="Output ONNX path.",
+    )
+    parser.add_argument("--chunk-frames", type=int, default=100, help="Fixed mel chunk length.")
+    parser.add_argument(
+        "--dtype",
+        type=str,
+        default="float32",
+        choices=["float16", "bfloat16", "float32"],
+        help="Torch dtype used for loading and export.",
+    )
+    parser.add_argument("--device", type=str, default="cpu", help="Torch device for export.")
+    return parser.parse_args()
+
+
+def main():
+    args = parse_args()
+    savepath = Path(args.savepath)
+    savepath.parent.mkdir(parents=True, exist_ok=True)
+
+    tower = load_audio_encoder(model_path=args.model_path, dtype=args.dtype, device=args.device)
+    wrapper = StaticChunkAudioEncoder(tower=tower, chunk_frames=args.chunk_frames).to(args.device).eval()
+
+    input_features = torch.zeros(
+        (1, 128, args.chunk_frames),
+        dtype=get_torch_dtype(args.dtype),
+        device=args.device,
+    )
+    feature_len = torch.tensor([args.chunk_frames], dtype=torch.int32, device=args.device)
+
+    with torch.no_grad():
+        torch_features, torch_valid_len = wrapper(input_features, feature_len)
+
+    torch.onnx.export(
+        wrapper,
+        (input_features, feature_len),
+        savepath.as_posix(),
+        input_names=["input_features", "feature_len"],
+        output_names=["audio_features", "valid_len"],
+        opset_version=18,
+        dynamo=False,
+        dynamic_axes={
+            "input_features": {0: "batch"},
+            "feature_len": {0: "batch"},
+            "audio_features": {0: "batch"},
+            "valid_len": {0: "batch"},
+        },
+    )
+
+    ort_session = ort.InferenceSession(savepath.as_posix(), providers=["CPUExecutionProvider"])
+    ort_features, ort_valid_len = ort_session.run(
+        None,
+        {
+            "input_features": input_features.detach().cpu().numpy(),
+            "feature_len": feature_len.detach().cpu().numpy(),
+        },
+    )
+    max_diff = float((torch_features.detach().cpu() - torch.from_numpy(ort_features)).abs().max().item())
+
+    print(f"saved: {savepath}")
+    print(f"chunk_frames: {args.chunk_frames}")
+    print(f"chunk_output_frames: {wrapper.max_aftercnn_len}")
+    print(f"torch_valid_len: {int(torch_valid_len[0].item())}")
+    print(f"ort_valid_len: {int(ort_valid_len.reshape(-1)[0])}")
+    print(f"max_abs_diff(torch_vs_onnx): {max_diff:.8f}")
+
+
+if __name__ == "__main__":
+    main()

convert/audio_encoder/export_audio_encoder_rknn.py

@@ -0,0 +1,56 @@
+import argparse
+from pathlib import Path
+
+import numpy as np
+from rknn.api import RKNN
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description="Convert Qwen3-ASR audio encoder ONNX to RKNN.")
+    parser.add_argument(
+        "--onnx-path",
+        type=str,
+        default="rknn_deploy/audio_encoder/onnx/qwen3_asr_audio_chunk100.onnx",
+        help="Path to exported ONNX model.",
+    )
+    parser.add_argument("--target-platform", type=str, default="rk3588", help="RK target platform.")
+    parser.add_argument("--chunk-frames", type=int, default=100, help="Fixed mel chunk length.")
+    parser.add_argument(
+        "--savepath",
+        type=str,
+        default="rknn_deploy/audio_encoder/rknn/qwen3_asr_audio_chunk100_rk3588.rknn",
+        help="Output RKNN path.",
+    )
+    return parser.parse_args()
+
+
+def main():
+    args = parse_args()
+    savepath = Path(args.savepath)
+    savepath.parent.mkdir(parents=True, exist_ok=True)
+
+    rknn = RKNN(verbose=False)
+    rknn.config(target_platform=args.target_platform)
+
+    ret = rknn.load_onnx(
+        model=args.onnx_path,
+        inputs=["input_features", "feature_len"],
+        input_size_list=[[1, 128, args.chunk_frames], [1]],
+        input_initial_val=[None, np.asarray([args.chunk_frames], dtype=np.int32)],
+    )
+    if ret != 0:
+        raise SystemExit(f"rknn.load_onnx failed: {ret}")
+
+    ret = rknn.build(do_quantization=False, dataset=None)
+    if ret != 0:
+        raise SystemExit(f"rknn.build failed: {ret}")
+
+    ret = rknn.export_rknn(savepath.as_posix())
+    if ret != 0:
+        raise SystemExit(f"rknn.export_rknn failed: {ret}")
+
+    print(f"saved: {savepath}")
+
+
+if __name__ == "__main__":
+    main()

convert/audio_encoder/onnx_run_audio_encoder.py

@@ -0,0 +1,73 @@
+import argparse
+from pathlib import Path
+
+import numpy as np
+import onnxruntime as ort
+
+from common import extract_mel_features, get_chunk_output_length_value, run_chunked_torch, split_mel_features
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description="Run Qwen3-ASR audio encoder ONNX by chunk and optionally compare to PyTorch.")
+    parser.add_argument("--model-path", type=str, default=".", help="Path to Qwen3-ASR model directory.")
+    parser.add_argument(
+        "--onnx-path",
+        type=str,
+        default="rknn_deploy/audio_encoder/onnx/qwen3_asr_audio_chunk100.onnx",
+        help="Path to exported ONNX.",
+    )
+    parser.add_argument("--audio-path", type=str, required=True, help="Path to input audio.")
+    parser.add_argument("--chunk-frames", type=int, default=100, help="Fixed mel chunk length.")
+    parser.add_argument("--compare-torch", action="store_true", help="Compare concatenated ONNX output with PyTorch.")
+    parser.add_argument("--savepath", type=str, default=None, help="Optional path to save concatenated features as .npy.")
+    return parser.parse_args()
+
+
+def main():
+    args = parse_args()
+    input_features, feature_len = extract_mel_features(args.model_path, args.audio_path)
+    chunks = split_mel_features(input_features, feature_len, args.chunk_frames)
+
+    session = ort.InferenceSession(args.onnx_path, providers=["CPUExecutionProvider"])
+
+    outputs = []
+    for chunk, chunk_len in chunks:
+        session_outputs = session.run(
+            None,
+            {
+                "input_features": chunk[None, ...].astype(np.float32),
+                "feature_len": np.asarray([chunk_len], dtype=np.int32),
+            },
+        )
+        ort_features = session_outputs[0]
+        if len(session_outputs) >= 2:
+            valid_len = int(np.asarray(session_outputs[1]).reshape(-1)[0])
+        else:
+            valid_len = get_chunk_output_length_value(chunk_len)
+        outputs.append(ort_features[0, :valid_len])
+
+    concat_features = np.concatenate(outputs, axis=0) if outputs else np.zeros((0, 2048), dtype=np.float32)
+    print(f"input_feature_len: {feature_len}")
+    print(f"num_chunks: {len(chunks)}")
+    print(f"audio_features: {concat_features.shape}")
+
+    if args.savepath:
+        savepath = Path(args.savepath)
+        savepath.parent.mkdir(parents=True, exist_ok=True)
+        np.save(savepath, concat_features)
+        print(f"saved_features: {savepath}")
+
+    if args.compare_torch:
+        ref = run_chunked_torch(
+            model_path=args.model_path,
+            input_features=input_features,
+            feature_len=feature_len,
+            chunk_frames=args.chunk_frames,
+        )
+        max_diff = float(np.max(np.abs(ref - concat_features))) if ref.size and concat_features.size else 0.0
+        print(f"torch_features: {ref.shape}")
+        print(f"max_abs_diff(torch_vs_onnx): {max_diff:.8f}")
+
+
+if __name__ == "__main__":
+    main()

convert/llm/export_rkllm_direct.py

@@ -0,0 +1,159 @@
+import argparse
+import os
+import sys
+from pathlib import Path
+
+from rkllm.api import RKLLM
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="Directly convert a Hugging Face model directory to RKLLM."
+    )
+    parser.add_argument(
+        "--model-path",
+        type=str,
+        default=".",
+        help="Path to the Hugging Face model directory.",
+    )
+    parser.add_argument(
+        "--target-platform",
+        type=str,
+        default="rk3588",
+        help="RK target platform.",
+    )
+    parser.add_argument(
+        "--num-npu-core",
+        type=int,
+        default=3,
+        help="Number of NPU cores to use during conversion.",
+    )
+    parser.add_argument(
+        "--device",
+        type=str,
+        default="cpu",
+        help="Device used by rkllm-toolkit during conversion, usually cpu or cuda.",
+    )
+    parser.add_argument(
+        "--dtype",
+        type=str,
+        default="float16",
+        choices=["float16", "bfloat16", "float32"],
+        help="Model loading dtype for rkllm-toolkit.",
+    )
+    parser.add_argument(
+        "--do-quantization",
+        action="store_true",
+        help="Enable RKLLM quantization. Disabled by default for FP16 export.",
+    )
+    parser.add_argument(
+        "--quantized-dtype",
+        type=str,
+        default="w8a8",
+        help="Quantized dtype used only when --do-quantization is set.",
+    )
+    parser.add_argument(
+        "--quantized-algorithm",
+        type=str,
+        default="normal",
+        help="Quantization algorithm used only when --do-quantization is set.",
+    )
+    parser.add_argument(
+        "--dataset",
+        type=str,
+        default=None,
+        help="Calibration dataset JSON path, only needed when --do-quantization is set.",
+    )
+    parser.add_argument(
+        "--optimization-level",
+        type=int,
+        default=1,
+        choices=[0, 1],
+        help="0 prioritizes speed/memory, 1 prioritizes accuracy.",
+    )
+    parser.add_argument(
+        "--max-context",
+        type=int,
+        default=4096,
+        help="Max context passed to rkllm build. Must be a multiple of 32.",
+    )
+    parser.add_argument(
+        "--savepath",
+        type=str,
+        default=None,
+        help="Output .rkllm path. If omitted, a default name is generated under ./rkllm.",
+    )
+    return parser.parse_args()
+
+
+def make_default_savepath(model_path: str, target_platform: str, do_quantization: bool, quantized_dtype: str) -> str:
+    model_name = Path(model_path).resolve().name or "model"
+    suffix = quantized_dtype if do_quantization else "fp16"
+    return os.path.join("rkllm", f"{model_name}_{suffix}_{target_platform}.rkllm")
+
+
+def main():
+    args = parse_args()
+
+    if args.max_context % 32 != 0:
+        raise ValueError("--max-context must be a multiple of 32")
+
+    if args.do_quantization and not args.dataset:
+        raise ValueError("--dataset is required when --do-quantization is enabled")
+
+    savepath = args.savepath or make_default_savepath(
+        model_path=args.model_path,
+        target_platform=args.target_platform,
+        do_quantization=args.do_quantization,
+        quantized_dtype=args.quantized_dtype,
+    )
+    os.makedirs(os.path.dirname(savepath) or ".", exist_ok=True)
+
+    print(f"[RKLLM] model_path={args.model_path}")
+    print(f"[RKLLM] target_platform={args.target_platform}")
+    print(f"[RKLLM] device={args.device}, dtype={args.dtype}")
+    print(f"[RKLLM] do_quantization={args.do_quantization}")
+    if args.do_quantization:
+        print(f"[RKLLM] quantized_dtype={args.quantized_dtype}")
+        print(f"[RKLLM] quantized_algorithm={args.quantized_algorithm}")
+        print(f"[RKLLM] dataset={args.dataset}")
+    print(f"[RKLLM] savepath={savepath}")
+
+    llm = RKLLM()
+
+    ret = llm.load_huggingface(
+        model=args.model_path,
+        device=args.device,
+        dtype=args.dtype,
+    )
+    if ret != 0:
+        print("[RKLLM] load_huggingface failed", file=sys.stderr)
+        raise SystemExit(ret)
+
+    build_kwargs = dict(
+        do_quantization=args.do_quantization,
+        optimization_level=args.optimization_level,
+        target_platform=args.target_platform,
+        num_npu_core=args.num_npu_core,
+        dataset=args.dataset,
+        max_context=args.max_context,
+    )
+    if args.do_quantization:
+        build_kwargs["quantized_dtype"] = args.quantized_dtype
+        build_kwargs["quantized_algorithm"] = args.quantized_algorithm
+
+    ret = llm.build(**build_kwargs)
+    if ret != 0:
+        print("[RKLLM] build failed", file=sys.stderr)
+        raise SystemExit(ret)
+
+    ret = llm.export_rkllm(savepath)
+    if ret != 0:
+        print("[RKLLM] export_rkllm failed", file=sys.stderr)
+        raise SystemExit(ret)
+
+    print(f"[RKLLM] export done: {savepath}")
+
+
+if __name__ == "__main__":
+    main()

convert/llm/extract_qwen3_text_model.py

@@ -0,0 +1,149 @@
+import argparse
+import json
+import shutil
+from pathlib import Path
+
+from safetensors import safe_open
+from safetensors.torch import save_file
+from transformers import Qwen3Config
+
+
+COPY_FILES = [
+    "generation_config.json",
+    "tokenizer_config.json",
+    "tokenizer.json",
+    "vocab.json",
+    "merges.txt",
+    "chat_template.json",
+    "special_tokens_map.json",
+    "preprocessor_config.json",
+]
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="Extract Qwen3-ASR thinker text weights into a standard Qwen3ForCausalLM Hugging Face directory."
+    )
+    parser.add_argument(
+        "--model-path",
+        type=str,
+        default=".",
+        help="Path to the original Qwen3-ASR model directory.",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=str,
+        default="rkllm/qwen3_text_hf",
+        help="Path to the output Hugging Face directory for standard Qwen3.",
+    )
+    parser.add_argument(
+        "--dry-run",
+        action="store_true",
+        help="Only print the planned extraction result without writing files.",
+    )
+    return parser.parse_args()
+
+
+def map_key(src_key: str) -> str | None:
+    if src_key.startswith("thinker.model."):
+        return "model." + src_key[len("thinker.model.") :]
+    if src_key == "thinker.lm_head.weight":
+        return "lm_head.weight"
+    return None
+
+
+def load_json(path: Path):
+    with path.open("r", encoding="utf-8") as f:
+        return json.load(f)
+
+
+def make_qwen3_config(src_config: dict, src_dir: Path) -> Qwen3Config:
+    text_config = dict(src_config["thinker_config"]["text_config"])
+    default_keys = set(Qwen3Config().to_dict().keys())
+    filtered = {k: v for k, v in text_config.items() if k in default_keys}
+    filtered["architectures"] = ["Qwen3ForCausalLM"]
+    filtered["model_type"] = "qwen3"
+    # Qwen3-ASR reuses a 3-axis MRoPE-style config shape, but its text side feeds
+    # identical position ids across all 3 axes. For plain Qwen3/RKLLM export, this
+    # should be treated as standard RoPE instead of multimodal MRoPE.
+    filtered["rope_scaling"] = None
+
+    generation_config = {}
+    generation_path = src_dir / "generation_config.json"
+    if generation_path.exists():
+        generation_config = load_json(generation_path)
+
+    filtered["pad_token_id"] = generation_config.get("pad_token_id", 151643)
+    filtered["eos_token_id"] = 151645
+    filtered["bos_token_id"] = generation_config.get("pad_token_id", 151643)
+    return Qwen3Config(**filtered)
+
+
+def copy_support_files(src_dir: Path, dst_dir: Path):
+    for name in COPY_FILES:
+        src = src_dir / name
+        if src.exists():
+            shutil.copy2(src, dst_dir / name)
+
+
+def main():
+    args = parse_args()
+    src_dir = Path(args.model_path).resolve()
+    dst_dir = Path(args.output_dir).resolve()
+    index_path = src_dir / "model.safetensors.index.json"
+    config_path = src_dir / "config.json"
+
+    src_index = load_json(index_path)
+    src_config = load_json(config_path)
+
+    selected = {}
+    total_size = 0
+    shard_to_keys: dict[str, list[tuple[str, str]]] = {}
+
+    for src_key, shard_name in src_index["weight_map"].items():
+        dst_key = map_key(src_key)
+        if dst_key is None:
+            continue
+        selected[dst_key] = shard_name
+        shard_to_keys.setdefault(shard_name, []).append((src_key, dst_key))
+
+    print(f"source_model: {src_dir}")
+    print(f"output_dir: {dst_dir}")
+    print(f"selected_tensors: {len(selected)}")
+    print(f"source_shards: {len(shard_to_keys)}")
+
+    if args.dry_run:
+        for shard_name, pairs in sorted(shard_to_keys.items()):
+            print(f"{shard_name}: {len(pairs)} tensors")
+        return
+
+    dst_dir.mkdir(parents=True, exist_ok=True)
+
+    for shard_name, pairs in sorted(shard_to_keys.items()):
+        shard_path = src_dir / shard_name
+        tensors = {}
+        with safe_open(shard_path, framework="pt", device="cpu") as f:
+            for src_key, dst_key in pairs:
+                tensor = f.get_tensor(src_key)
+                tensors[dst_key] = tensor
+                total_size += tensor.numel() * tensor.element_size()
+        save_file(tensors, str(dst_dir / shard_name))
+        print(f"wrote {shard_name}: {len(tensors)} tensors")
+
+    dst_index = {
+        "metadata": {"total_size": total_size},
+        "weight_map": {dst_key: shard_name for dst_key, shard_name in selected.items()},
+    }
+    with (dst_dir / "model.safetensors.index.json").open("w", encoding="utf-8") as f:
+        json.dump(dst_index, f, ensure_ascii=False, indent=2)
+        f.write("\n")
+
+    qwen3_config = make_qwen3_config(src_config, src_dir)
+    qwen3_config.save_pretrained(dst_dir)
+
+    copy_support_files(src_dir, dst_dir)
+    print("done")
+
+
+if __name__ == "__main__":
+    main()