bne-float-384

Float32 baseline for the Binary Native Embeddings project.

• Backbone: prajjwal1/bert-mini (4L × 256d, ~11M params)
• Output: 384-dim float32 via Linear(256→384) + mean pooling
• Training: MultipleNegativesRankingLoss on NLI 550k pairs, 3 epochs

STS-B Spearman	Recall@10 (SciFact)	Memory / 1k vecs
0.7355	0.3131	1.46 MB

Part of binary-native-embeddings-for-CPU-Retrieval · Discussion

Usage

import torch
from transformers import BertTokenizer
from huggingface_hub import hf_hub_download

tokenizer = BertTokenizer.from_pretrained("prajjwal1/bert-mini")

# load model
from models.float_embedder import FloatEmbedder
model = FloatEmbedder(output_dim=384)
weights = hf_hub_download("korben99/bne-float-384", "float_embedder.pt")
model.load_state_dict(torch.load(weights, map_location="cpu"))
model.eval()

vecs = model.encode(["hello world"], tokenizer)  # (1, 384) float32