How to use from the
Use from the
Model2Vec library
from model2vec import StaticModel

model = StaticModel.from_pretrained("MisasAI/LittlePotion-1BPW")

embeddings = model.encode(["It's dangerous to go alone!", "It's a secret to everybody."])
print(embeddings.shape)

LittlePotion-1BPW 931kb

potion-base-8M squeezed down to 1 bit per weight, from 15 MB to 931 KB, using LittleBit, a compression method from Samsung Research.

Want it even smaller? See LittleLittlePotion-164kb. And see how this was done.

This is an unofficial version and all credit goes to MinishLab for the original.

1BPW is impossible

Nope, JPEG manages similar compression. And actually the method is very similar to JPEG compression.

How it works: potion is basically one big table, one row of numbers per word piece. LittleBit rewrites that table as two much smaller tables made only of +1 and -1 (one bit each), plus a few scaling numbers. Squashing it like that breaks it at first, so it's retrained afterwards (on Wikipedia text, matching the similarities between sentences) until it gives nearly the same answers as the original.

How good is it?

Size STS-B SICK-R SciFact
original potion-base-8M 15 MB 0.754 0.647 0.505
LittlePotion-1BPW 931 KB 0.737 0.625 0.453
kept 16x smaller 98% 97% 90%

STS-B and SICK-R test "are these two sentences about the same thing?" (Spearman). SciFact tests search (nDCG@10). Search drops more because it was only retrained for similarity.

If you don't need it this small, normal 4-bit potion (about 4 MB) is basically lossless and simpler. This is just an interesting proof of concept model.

What's in here

File What it is
littlepotion-931kb.lbit the model (931 KB)
tokenizer.json potion's tokenizer
littlepotion.py small loader
benchmark.py checks the numbers below yourself
scripts/potion_lb.py the script that made it

How to use it

from huggingface_hub import snapshot_download
import sys

path = snapshot_download("MisasAI/LittlePotion-1BPW")
sys.path.insert(0, path)
from littlepotion import load

model = load(f"{path}/littlepotion-931kb.lbit")
vecs = model.encode(["The cat sat on the mat.", "A kitten rested on the rug."])
print(vecs @ vecs.T)   # higher = more similar

You need torch, safetensors and tokenizers. If model2vec is installed you get a normal StaticModel back, otherwise a small stand-in with the same encode().

Check it yourself

pip install -r requirements.txt
python benchmark.py

Make your own

scripts/potion_lb.py compresses and retrains potion-base-8M at any bit rate:

python scripts/potion_lb.py --bpw 1.0 --steps 3000

Credits

  • potion-base-8M and Model2Vec by Stephan Tulkens and Thomas van Dongen, MinishLab (MIT). LittlePotion is just their work, compressed.
  • LittleBit by Lee, Kim, You and Kim, Samsung Research, NeurIPS 2025 (arXiv:2506.13771). This is an independent implementation from the paper, not Samsung's code.
@software{minishlab2024model2vec,
  author    = {Stephan Tulkens and {van Dongen}, Thomas},
  title     = {Model2Vec: Fast State-of-the-Art Static Embeddings},
  year      = {2024},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.17270888},
  url       = {https://github.com/MinishLab/model2vec},
  license   = {MIT}
}

@inproceedings{lee2025littlebit,
  author    = {Banseok Lee and Dongkyu Kim and Youngcheon You and Youngmin Kim},
  title     = {LittleBit: Ultra Low-Bit Quantization via Latent Factorization},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2025},
  eprint    = {2506.13771},
  archivePrefix = {arXiv}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for MisasAI/LittlePotion-1BPW

Quantized
(2)
this model

Dataset used to train MisasAI/LittlePotion-1BPW

Paper for MisasAI/LittlePotion-1BPW