Veyra-Embed-125M: Sentence Similarity Embedding Model

Veyra-Embed-125M is a from-scratch sentence embedding model developed by Dl26. It maps text into normalized dense vectors for sentence similarity, semantic search, clustering, and retrieval-style experiments.

The model is trained with a contrastive objective over real sentence pairs using in-batch negatives. It is designed as a compact encoder-style embedding model rather than a generative language model.

Model Details

Property	Value
Developer	Dl26
Model type	Sentence embedding encoder
Parameters	125,560,320
Hidden size	768
Layers	4
Tokenizer vocab size	32,430
Model vocab size	135,000
Embedding size	768
Max length	64
Objective	Symmetric contrastive InfoNCE
Training data	sentence-transformers/all-nli, sentence-transformers/stsb, sentence-transformers/quora-duplicates, embedding-data/QQP_triplets

Usage

This checkpoint contains raw PyTorch/safetensors weights plus tokenizer files. A compatible implementation should create the same encoder architecture from config.json, load model.safetensors, then mean-pool and normalize the output embedding.

from safetensors.torch import load_file

state_dict = load_file("model.safetensors")

Intended Use

• sentence similarity
• semantic search
• duplicate question retrieval
• clustering
• lightweight embedding research
• ranking experiments

Limitations

• It is not a generative language model.
• It should be evaluated on target retrieval and similarity datasets before use.
• It will not match large production embedding models trained on much larger curated mixtures.