zero v1 — M7 lookup table, run p35w-2m-s2500
Browse files
README.md
CHANGED
|
@@ -72,6 +72,39 @@ print(docs[int(np.argmax(scores))])
|
|
| 72 |
That asymmetry is the point: `doc_model` runs once per document, in the cloud. `enc` runs on
|
| 73 |
every query, on the device, and costs almost nothing.
|
| 74 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
### The rule, if you reimplement it
|
| 76 |
|
| 77 |
Tokenize with the bundled WordPiece tokenizer (`add_special_tokens=True`, truncate at 512,
|
|
|
|
| 72 |
That asymmetry is the point: `doc_model` runs once per document, in the cloud. `enc` runs on
|
| 73 |
every query, on the device, and costs almost nothing.
|
| 74 |
|
| 75 |
+
## Using it with Qdrant
|
| 76 |
+
|
| 77 |
+
The output is an ordinary 1024-d dense vector, so no special handling is needed. Both sides are
|
| 78 |
+
L2-normalized, which means `DOT` ranks identically to `COSINE` and is cheaper.
|
| 79 |
+
|
| 80 |
+
```python
|
| 81 |
+
from qdrant_client import QdrantClient, models
|
| 82 |
+
|
| 83 |
+
client = QdrantClient(":memory:") # or your cluster
|
| 84 |
+
client.create_collection("docs", vectors_config=models.VectorParams(
|
| 85 |
+
size=1024, distance=models.Distance.DOT))
|
| 86 |
+
client.upsert("docs", points=[
|
| 87 |
+
models.PointStruct(id=i, vector=D[i].tolist(), payload={"text": t})
|
| 88 |
+
for i, t in enumerate(docs)]) # D from the document encoder above
|
| 89 |
+
|
| 90 |
+
hits = client.query_points("docs", query=enc.encode([q])[0].tolist(), limit=5).points
|
| 91 |
+
```
|
| 92 |
+
|
| 93 |
+
**On the edge, put the table in the store too.** A second collection holds one point per vocab
|
| 94 |
+
row, created with `hnsw_config=models.HnswConfigDiff(m=0)` — it is retrieve-by-id only, and
|
| 95 |
+
indexing it inflated the shard from 466 MB to 1.82 GB for no benefit. The query path becomes
|
| 96 |
+
tokenize → fetch rows by id → pool → search, with no model weights in your process at all.
|
| 97 |
+
|
| 98 |
+
**If you fuse with BM25, the fusion rule matters.** The system that ties OpenSearch (0.4911)
|
| 99 |
+
uses **convex score fusion at w=0.8**, not RRF: on development sets RRF scored 0.5504 against
|
| 100 |
+
convex's 0.5727. Qdrant's native `Fusion.RRF` is therefore a *different, weaker* operating point,
|
| 101 |
+
not the published one — combine the scores yourself to reproduce it. Dense-only in Qdrant
|
| 102 |
+
reproduces 0.4339 exactly.
|
| 103 |
+
|
| 104 |
+
Scalar-quantizing the document index to int8 halves it to ~1.02 GB per 1M vectors, but **that
|
| 105 |
+
was never measured for quality here** — treat it as untested. (The int8 quality-free result
|
| 106 |
+
above is about the *query table*, not the document index.)
|
| 107 |
+
|
| 108 |
### The rule, if you reimplement it
|
| 109 |
|
| 110 |
Tokenize with the bundled WordPiece tokenizer (`add_special_tokens=True`, truncate at 512,
|