FunAudioLLM/SenseVoiceSmall, alternative projection and head weights prepared for Tenstorrent Tensix NPUs (Wormhole, Blackhole).
- Done for Tensix: 280 encoder projections in
bfloat4_b(BFP4, 4.5 packed bits a value) and 281 projection and CTC-head operands inbfloat8_b(BFP8, 8.5 packed bits a value), quantized by round to nearest from bf16, plus 281 bf16 alternatives; all stored as bf16. - Size: 837,872,640 bytes packed for all three groups; an integration picks only the alternatives it needs.
- Checked: All 842 tensors are finite and exactly match the known query scaling followed by BFP4 or BFP8 quantization from bf16, or bf16 conversion. The archive has no task score.
- Code: TT-NN 0.81.1 in a simulator, where all 842 tensors repack exactly (its
tools/pack_tensix.pyandtools/verify_grid.pyship here unchanged); tensix-bfp-gptq v0.1.0 (quantization and format checks), tensix-kernels v0.1.0 and tensix-runner v0.1.0 (kernels and runner for Tensix).
Existing projection and head alternatives derived from SenseVoiceSmall, decoded into portable safetensors by Xylolabs with all saved values preserved. The three groups belong to the same BF16-first preparation route; an older BFP8 component archive has different quantized matrix values.
Scope
This partial archive contains 280 BFP4 encoder projections, 281 BFP8 encoder-projection/CTC-head operands and 281 BF16 projection/head alternatives. Its source profile has 128 feature-input rows, with four fixed Korean/event/emotion/ITN-on query rows. To retain the saved BFP4 profile, use the BFP8 CTC head supplied here. Architecture integration, compatible shared operands, preprocessing and token decoding are required.
The companion archive provides shared normalization, bias, FSMN/tap and static operands. Its BFP8 matrices came through a different rounding route; use the alternatives in this archive when preserving the BF16-first route. A mixed selection needs its own integrated recognizer evaluation.
Formats
| Component | Tensors | Native interpretation | Portable file bytes |
|---|---|---|---|
bf4_projections |
280 | BFP4 round-to-nearest after BF16; 4.5 packed bits/value | 440,576,912 |
bf8_projections |
281 | BFP8 round-to-nearest after BF16; 8.5 packed bits/value | 466,233,312 |
bf16_projections |
281 | BF16 | 466,233,312 |
All three portable files store BF16. BFP4 and BFP8 files retain already-quantized values exactly; their download sizes do not describe packed density. Counting all alternatives together, native tiled data totals 837,872,640 bytes. An integration selects compatible alternatives and need not keep all full groups resident.
Matrices retain their saved [in, out] operator order. Query scaling is folded in already. component.json
records each hash, shape, stored dtype and native-format label.
Quickstart
Install PyTorch, safetensors and huggingface_hub, then load one component at a time:
from pathlib import Path
import importlib.util
from huggingface_hub import snapshot_download
folder = Path(snapshot_download("Xylolabs/SenseVoiceSmall-Tenstorrent-Tensix-Precision-Components", allow_patterns=[
"component.json", "load_components.py", "bf4_projections.safetensors",
"README.md", "LICENSE", "NOTICE", "CODE_LICENSE",
]))
spec = importlib.util.spec_from_file_location("sensevoice_precision", folder / "load_components.py")
components = importlib.util.module_from_spec(spec)
spec.loader.exec_module(components)
count = sum(1 for name, tensor in components.iter_component(folder, "bf4_projections"))
print(count)
This checks the hash, names, shapes, declared dtype and finite values of 280 matrices while streaming. Load bf8_projections or bf16_projections and the corresponding file for their 281 projection/head alternatives. Integration must preserve saved matrix order, folded query scaling, compatible shared operands and the selected native packing format.
Evaluation
All 842 saved tensors are finite, decoded without changing values, and repack exactly through TT-NN 0.81.1 in a simulator context. The original ONNX matches its primary Hugging Face LFS identity, its 919 source arrays match a fresh reconstruction, and all 842 alternatives exactly match the existing query-scaling and BF16-first BFP4/BFP8 or BF16 transformations. Transport checksums cover every original cache file. The streaming public example and all three descriptor groups are checked.
These are representation and provenance checks. An integrated recognizer requires speech evaluation for its selected operands; no task score is assigned to the standalone archive.
License
Weights retain the full FunASR Model Open Source License Agreement v1.1, as linked by the original
SenseVoiceSmall release. It permits copying, modification and sharing under its terms, including source/author
attribution and model-name retention. Reference/learning-purpose, community-conduct, termination and revision
provisions remain in the agreement. NOTICE identifies FunAudioLLM, Alibaba Group, the sherpa-onnx source and
our modifications. Only the Xylolabs loader uses Apache 2.0 (CODE_LICENSE). Provided as is, without
warranty. Not affiliated with or endorsed by Tenstorrent.
Citation
Cite the original SenseVoiceSmall release and the sherpa-onnx export.
Model tree for Xylolabs/SenseVoiceSmall-Tenstorrent-Tensix-Precision-Components
Base model
FunAudioLLM/SenseVoiceSmall