metadata
license: apache-2.0
embeddings
This is a repo created to keep a collection of quantized bert models in ggml format.
usage
You can utilize bert.cpp as usual or use our new api to quickly prototype on real use-case scenarios for text similarity.
| model | api | mb |
|---|---|---|
| l3 | nano | 11.2 |
| l6 | small | 14.5 |
| l12 | ❌ | 21.3 |
| bge-small | medium | 21.3 |
| bge-base | large | 68.8 |
| bge-large | ❌ | 200 |
We are planning to update the list to always support lastest open-source models.
embeddings sample
./main -m small -p "word"
[0.0698, -0.0024, -0.0153, 0.0193, -0.1060, -0.0278, 0.1424, -0.0056, -0.0536...
api sample
{ ...,
"similarity": {
"semantic": 0.035270897102758283,
"literal": 0.22219999999999998
}
}