wav2vec2-bert-tiny2-s-emotion-russian-resd · quantized

Quantized builds of Aniemore/wav2vec2-bert-tiny2-s-emotion-russian-resd — speech emotion recognition for Russian over seven classes: anger, disgust, enthusiasm, fear, happiness, neutral, sadness.

The weights here are the published original, quantized. They were not retrained, and they are not a different model: on RESD test every variant lands within the seed spread of the original.

Variants

subfolder	scheme	weights	vs fp32	macro-F1	UA	WA
(original repo)	fp32	1318 MiB	1.0x	0.7228	0.7217	0.7286
`int8`	W8A16	463 MiB	2.8x smaller	0.7271	0.7261	0.7321
`fp8`	W8A16-float	448 MiB	2.9x smaller	0.7341	0.7332	0.7393
`int4`	W4A16_ASYM	320 MiB	4.1x smaller	0.7168	0.7147	0.7214

Usage

Pick a variant with subfolder=. Only that subfolder is downloaded.

import torch, librosa
from transformers import AutoModelForAudioClassification, AutoFeatureExtractor

repo = "Aniemore/wav2vec2-bert-tiny2-s-emotion-russian-resd-quantized"
model = AutoModelForAudioClassification.from_pretrained(
    repo, subfolder="int8").eval()          # or "fp8", "int4"
fe = AutoFeatureExtractor.from_pretrained(repo, subfolder="int8")

# Resample to 16 kHz. Do not skip it: RESD itself ships at 44.1 kHz,
# and handing the model 44.1 kHz audio while telling the extractor it
# is 16 kHz stretches time 2.8x and silently changes the answer.
wav, _ = librosa.load("clip.wav", sr=16000, mono=True)
x = fe(wav, sampling_rate=16000, return_tensors="pt", padding=True)
with torch.no_grad():
    probs = model(**x).logits.softmax(-1)[0]
print({model.config.id2label[i]: round(p.item(), 3) for i, p in enumerate(probs)})

Loading the audio without librosa

# torchaudio
import torchaudio
wav, sr = torchaudio.load("clip.wav")
wav = torchaudio.functional.resample(wav, sr, 16000).mean(0).numpy()

# torchcodec, the newer decoder
from torchcodec.decoders import AudioDecoder
wav = AudioDecoder("clip.wav", sample_rate=16000).get_all_samples().data.mean(0).numpy()

# straight from the dataset, which resamples on the column
from datasets import load_dataset, Audio
ds = load_dataset("Aniemore/resd", split="test")
ds = ds.cast_column("speech", Audio(sampling_rate=16000))
wav = ds[0]["speech"]["array"]

What is quantized

Only nn.Linear inside the encoder blocks — that is where 98.6% of the weight mass sits. Left in fp32: the convolutional feature extractor, the positional convolution, the layer norms, the projector and the classifier. The classifier is seven rows wide, so quantizing it would save nothing and put rounding error straight onto the logits.

int8 — W8A16, 8-bit integer weights, group size 128, symmetric. Activations are not quantized.
fp8 — W8A16-float, 8-bit float8_e4m3 weights, per output channel, symmetric. Activations are not quantized.
int4 — W4A16_ASYM, 4-bit integer weights, group size 128, asymmetric. Activations are not quantized.

Format is compressed-tensors; transformers loads it directly, no extra package.

Whether GPU memory drops depends on the runtime

The packed weights are smaller wherever they stay packed. What varies is whether the loader keeps them that way:

vLLM and other compressed-tensors-aware runtimes keep the weights packed and dequantize per tile inside the kernel. Memory drops roughly with the table above, and W4A16 hits the Marlin path on Ampere and newer.
Plain transformers decompresses to the compute dtype while loading, so the resident model is the size fp32 or bf16 would be. Load with dtype=torch.bfloat16 if you want the memory back on this path.

So the download and the checkpoint always shrink; resident GPU memory shrinks when the runtime can execute the packed format directly.

On FP8 specifically: the arithmetic exists from Ada and Hopper onward (RTX 40-series, L4, H100). On Ampere there is no fp8 datapath, so an fp8 checkpoint is upconverted to run — it still saves the download, but on those cards int8 or int4 is the variant that pays off.

Behaviour of the underlying model

Evaluation

RESD test, 280 clips, seven classes. Audio is resampled to 16 kHz mono, clips capped at 12 s, normalized per utterance, padding masked. UA is macro-averaged recall, WA is accuracy, F1 is macro-averaged. Every variant went through the same harness as the original, so the numbers are directly comparable.

The split matches fold 1 of EmoBox bit for bit, so these numbers can be read against that leaderboard — with the caveat that the training protocol differs.

RESD is acted and balanced; real speech is not. A score here does not transfer to spontaneous audio, where the neutral class dominates. Measure on your own material before deploying.