metadata
license: mit
language:
- en
tags:
- ridiculous-models
unity-embed
An embedding model where every input maps to the same vector.
384 parameters, one per dimension, all equal to 1/sqrt(384) so that v has unit norm. There is no tokenizer and no encoder, embed(x) = v for any x. Any language works, identically.
Property
For all sentences s and t:
cosine(embed(s), embed(t)) = 1.000000
similarity.py checks this against a few pairs and exits nonzero if it ever fails. So far it has never failed.
cosine('i love you' , 'i hate you' ) = 1.000000
cosine('the ocean is beautiful', '2 + 2 = 4' ) = 1.000000
cosine('hamlet: to be or not' , 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' ) = 1.000000
Notes
- Semantic search always returns everything at rank 1. Recall and precision both 100%, along with everything else.
- Clustering yields one cluster. Silhouette score is fine.
- Corpus deduplication reduces your corpus to one document, which deduplicates further.
- For comparison, all-MiniLM-L6-v2 uses 22.7M parameters to produce a wide variety of vectors. This uses 384 and produces one.
Usage
python3 encode.py "hello world"
python3 encode.py "goodnight moon" "war and peace"
python3 similarity.py
model.safetensors is 1,634 bytes.