Xylolabs

SenseVoiceSmall precision alternatives for Tenstorrent Tensix

Base model Formats License Code Tenstorrent Tensix

FunAudioLLM/SenseVoiceSmall, alternative projection and head weights prepared for Tenstorrent Tensix NPUs (Wormhole, Blackhole).

  • Done for Tensix: 280 encoder projections in bfloat4_b (BFP4, 4.5 packed bits a value) and 281 projection and CTC-head operands in bfloat8_b (BFP8, 8.5 packed bits a value), quantized by round to nearest from bf16, plus 281 bf16 alternatives; all stored as bf16.
  • Size: 837,872,640 bytes packed for all three groups; an integration picks only the alternatives it needs.
  • Checked: All 842 tensors are finite and exactly match the known query scaling followed by BFP4 or BFP8 quantization from bf16, or bf16 conversion. The archive has no task score.
  • Code: TT-NN 0.81.1 in a simulator, where all 842 tensors repack exactly (its tools/pack_tensix.py and tools/verify_grid.py ship here unchanged); tensix-bfp-gptq v0.1.0 (quantization and format checks), tensix-kernels v0.1.0 and tensix-runner v0.1.0 (kernels and runner for Tensix).

Existing projection and head alternatives derived from SenseVoiceSmall, decoded into portable safetensors by Xylolabs with all saved values preserved. The three groups belong to the same BF16-first preparation route; an older BFP8 component archive has different quantized matrix values.

Scope

This partial archive contains 280 BFP4 encoder projections, 281 BFP8 encoder-projection/CTC-head operands and 281 BF16 projection/head alternatives. Its source profile has 128 feature-input rows, with four fixed Korean/event/emotion/ITN-on query rows. To retain the saved BFP4 profile, use the BFP8 CTC head supplied here. Architecture integration, compatible shared operands, preprocessing and token decoding are required.

The companion archive provides shared normalization, bias, FSMN/tap and static operands. Its BFP8 matrices came through a different rounding route; use the alternatives in this archive when preserving the BF16-first route. A mixed selection needs its own integrated recognizer evaluation.

Formats

Component Tensors Native interpretation Portable file bytes
bf4_projections 280 BFP4 round-to-nearest after BF16; 4.5 packed bits/value 440,576,912
bf8_projections 281 BFP8 round-to-nearest after BF16; 8.5 packed bits/value 466,233,312
bf16_projections 281 BF16 466,233,312

All three portable files store BF16. BFP4 and BFP8 files retain already-quantized values exactly; their download sizes do not describe packed density. Counting all alternatives together, native tiled data totals 837,872,640 bytes. An integration selects compatible alternatives and need not keep all full groups resident.

Matrices retain their saved [in, out] operator order. Query scaling is folded in already. component.json records each hash, shape, stored dtype and native-format label.

Quickstart

Install PyTorch, safetensors and huggingface_hub, then load one component at a time:

from pathlib import Path
import importlib.util
from huggingface_hub import snapshot_download

folder = Path(snapshot_download("Xylolabs/SenseVoiceSmall-Tenstorrent-Tensix-Precision-Components", allow_patterns=[
    "component.json", "load_components.py", "bf4_projections.safetensors",
    "README.md", "LICENSE", "NOTICE", "CODE_LICENSE",
]))
spec = importlib.util.spec_from_file_location("sensevoice_precision", folder / "load_components.py")
components = importlib.util.module_from_spec(spec)
spec.loader.exec_module(components)
count = sum(1 for name, tensor in components.iter_component(folder, "bf4_projections"))
print(count)

This checks the hash, names, shapes, declared dtype and finite values of 280 matrices while streaming. Load bf8_projections or bf16_projections and the corresponding file for their 281 projection/head alternatives. Integration must preserve saved matrix order, folded query scaling, compatible shared operands and the selected native packing format.

Evaluation

All 842 saved tensors are finite, decoded without changing values, and repack exactly through TT-NN 0.81.1 in a simulator context. The original ONNX matches its primary Hugging Face LFS identity, its 919 source arrays match a fresh reconstruction, and all 842 alternatives exactly match the existing query-scaling and BF16-first BFP4/BFP8 or BF16 transformations. Transport checksums cover every original cache file. The streaming public example and all three descriptor groups are checked.

These are representation and provenance checks. An integrated recognizer requires speech evaluation for its selected operands; no task score is assigned to the standalone archive.

License

Weights retain the full FunASR Model Open Source License Agreement v1.1, as linked by the original SenseVoiceSmall release. It permits copying, modification and sharing under its terms, including source/author attribution and model-name retention. Reference/learning-purpose, community-conduct, termination and revision provisions remain in the agreement. NOTICE identifies FunAudioLLM, Alibaba Group, the sherpa-onnx source and our modifications. Only the Xylolabs loader uses Apache 2.0 (CODE_LICENSE). Provided as is, without warranty. Not affiliated with or endorsed by Tenstorrent.

Citation

Cite the original SenseVoiceSmall release and the sherpa-onnx export.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Xylolabs/SenseVoiceSmall-Tenstorrent-Tensix-Precision-Components

Finetuned
(30)
this model

Collection including Xylolabs/SenseVoiceSmall-Tenstorrent-Tensix-Precision-Components