---
library_name: llima
license: apache-2.0
pipeline_tag: automatic-speech-recognition
base_model: openai/whisper-medium
base_model_relation: quantized
language:
- en
- zh
- de
- es
- ru
- ko
- fr
- ja
- pt
- tr
- pl
- ca
- nl
- ar
- sv
- it
- id
- hi
- fi
- vi
- he
- uk
- el
- ms
- cs
- ro
- da
- hu
- ta
- "no"
- th
- ur
- hr
- bg
- lt
- la
- mi
- ml
- cy
- sk
- te
- fa
- lv
- bn
- sr
- az
- sl
- kn
- et
- mk
- br
- eu
- is
- hy
- ne
- mn
- bs
- kk
- sq
- sw
- gl
- mr
- pa
- si
- km
- sn
- yo
- so
- af
- oc
- ka
- be
- tg
- sd
- gu
- am
- yi
- lo
- uz
- fo
- ht
- ps
- tk
- nn
- mt
- sa
- lb
- my
- bo
- tl
- mg
- as
- tt
- haw
- ln
- ha
- ba
- jw
- su
tags:
- whisper
- audio
- automatic-speech-recognition
- speech-translation
- embedded
- sima
- modalix
- a16w8
---

# Whisper Medium A16W8: Optimized for SiMa.ai Modalix

## Overview

This repository contains a precompiled and quantized version of
[OpenAI Whisper Medium](https://huggingface.co/openai/whisper-medium), optimized
for execution on the **SiMa.ai Modalix** platform.

| Property | Value |
|---|---|
| Model architecture | Whisper encoder-decoder Transformer |
| Parameters | 769 million |
| Quantization | A16W8: 16-bit activations, 8-bit weights |
| Target | SiMa.ai Modalix |
| Tasks | Multilingual transcription and speech translation to English |
| Languages | 99 |
| Maximum audio window | 30 seconds |
| Maximum decoder positions | 448 |
| Compatible Neat release | 0.4.0 |
| Source model | [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) |

> This repository is not a standard Transformers checkpoint. The `elf_files/`
> and `devkit/` directories contain compiled artifacts consumed by the SiMa.ai
> Neat runtime on Modalix.

## Prerequisites

To use this model, you need:

1. A **SiMa.ai Modalix device**.
2. The **SiMa.ai Neat Runtime** installed or updated on Modalix.
3. The **Hugging Face CLI**, optionally, for downloading the model on a host
   before copying it to Modalix.

Follow the [SiMa.ai Neat getting started guide](https://developer.sima.ai/software/getting-started/)
to install or update the Neat Runtime.

## Installation and Deployment

### Download the Model

On Modalix, download the precompiled model with the LLiMa model manager:

```bash
llima pull whisper-medium-a16w8
```

The model is installed under:

```text
/media/nvme/llima/models/whisper-medium-a16w8
```

Alternatively, download it on a host and copy it to Modalix:

```bash
hf download simaai/whisper-medium-a16w8 \
  --local-dir whisper-medium-a16w8

scp -r whisper-medium-a16w8 \
  sima@<modalix-ip>:/media/nvme/llima/models/
```

Replace `<modalix-ip>` with the IP address or hostname of the Modalix device.

## Usage

This model is intended for applications built with the SiMa.ai Neat GenAI
APIs. Use `ASRModel` for direct, in-process transcription or `GenAIServer` when
clients access the model over HTTP.

### Direct Application Integration

The Neat `ASRModel` API provides the lowest-overhead path for speech
transcription inside an application:

```cpp
#include "neat/genai.h"

#include <iostream>

int main() {
    simaai::neat::genai::ASRModel model(
        "/media/nvme/llima/models/whisper-medium-a16w8");

    simaai::neat::genai::GenerationRequest request;
    request.audio_file = "speech.wav";
    request.language = "en";

    auto result = model.run(request);
    std::cout << result.text << "\n";
}
```

For the complete C++ and Python workflows, including audio tensors and
streaming, see
[GenAI Model](https://developer.sima.ai/software/develop-apps/development-workflow/genai-model).

### Serve with the Neat GenAI Server

Use `GenAIServer` when a browser, service, or remote client needs to access the
model through an HTTP API. The packaged Neat tutorial can serve this ASR model
by itself:

```bash
./lib/sima-neat/tutorials/tutorial_021_serve_genai_models \
  --asr /media/nvme/llima/models/whisper-medium-a16w8
```

Verify that the model was registered:

```bash
curl http://<modalix-ip>:9998/v1/models
```

Send an audio transcription request with the tutorial client:

```bash
python3 share/sima-neat/tutorials/021_serve_genai_models/request_audio_transcription.py \
  --server-ip <modalix-ip> \
  --model asr \
  speech.wav
```

See
[Serve GenAI Models](https://developer.sima.ai/software/tutorials/serve-genai-models)
for the complete C++, Python, and HTTP workflow.

## Artifact Layout

```text
devkit/                         Runtime configuration, tokenizer, and embeddings
elf_files/                      Compiled Modalix accelerator programs
compile.sh                      Compilation entry point
gen_models--openai--whisper.py  Whisper compilation utility
```

The compilation scripts are included for provenance and reproducibility. They
are not required when using the precompiled artifacts.

## Evaluation

No accuracy or performance results specific to this A16W8 compiled artifact
are currently reported. Results on the
[source model card](https://huggingface.co/openai/whisper-medium) describe the
unquantized upstream checkpoint and should not be treated as measurements of
this build.

## Limitations

- These artifacts run on SiMa.ai Modalix and cannot be loaded with
  `transformers.AutoModel`.
- The runtime processes an audio window of up to 30 seconds. Segment longer
  recordings before inference.
- This build produces text without word- or segment-level timestamps.
- Speaker diarization is not provided.
- Quantization can cause minor differences from the full-precision source
  model.
- Whisper can hallucinate text during silence or noisy audio and performs
  unevenly across languages, accents, and domains. Evaluate the model on
  representative data before production deployment.

## Resources

- [SiMa.ai Neat](https://developer.sima.ai/software/getting-started/)
- [GenAI Model API](https://developer.sima.ai/software/develop-apps/development-workflow/genai-model)
- [Serve GenAI Models](https://developer.sima.ai/software/tutorials/serve-genai-models)
- [GenAI with LLiMa](https://developer.sima.ai/software/genai-llima/)
- [OpenAI Whisper Medium](https://huggingface.co/openai/whisper-medium)
- [Whisper paper](https://arxiv.org/abs/2212.04356)

## Citation

```bibtex
@misc{radford2022whisper,
  doi       = {10.48550/ARXIV.2212.04356},
  url       = {https://arxiv.org/abs/2212.04356},
  author    = {Radford, Alec and Kim, Jong Wook and Xu, Tao and
               Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  title     = {Robust Speech Recognition via Large-Scale Weak Supervision},
  publisher = {arXiv},
  year      = {2022}
}
```