potion-multilingual-128M, NF4

minishlab/potion-multilingual-128M at revision 73908c3438cf03b6a01bcb9611d62b23d0726f08, with its embedding table stored as 4-bit NF4 codes and one FP16 scale per 64 values. The model file is 65 MiB, against 244 MiB for FP16. The tokenizer and config are unchanged.

The file replaces the embeddings tensor with embeddings_q4 (U8, two codes per byte, low nibble first), embeddings_q4_scales (F16) and embeddings_q4_levels (F32, 16 levels). A value is levels[code] * scales[row, column // 64]. The model2vec Python package does not read this layout.

Built from minishlab/potion-multilingual-128M@73908c3438cf03b6a01bcb9611d62b23d0726f08 by quantize_model2vec.py at source commit 8a69676ebd3bcd03aa77bd09f18af6f964526dda.

Downloads last month: 118

Safetensors

Model size

66M params

Tensor type

F32

F16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for llamaindex/potion-multilingual-128M-nf4

Base model

minishlab/potion-multilingual-128M

Finetuned

(39)

this model