EverMind-AI commited on
Commit
e773f61
·
verified ·
1 Parent(s): 810ea8e

Define the query prefix and doc format before the snippet uses them

Browse files
Files changed (1) hide show
  1. README.md +14 -19
README.md CHANGED
@@ -36,7 +36,11 @@ which reranks this model's top candidates.
36
 
37
  ## Usage
38
 
39
- Embeddings are L2-normalized, so cosine similarity is a plain dot product.
 
 
 
 
40
 
41
  ```python
42
  import torch
@@ -47,6 +51,15 @@ MODEL = "EverMind-AI/skillcorpus-embedding-0.6b"
47
  tok = AutoTokenizer.from_pretrained(MODEL, padding_side="left")
48
  model = AutoModel.from_pretrained(MODEL, dtype=torch.bfloat16).cuda().eval()
49
 
 
 
 
 
 
 
 
 
 
50
 
51
  def last_token_pool(hidden, attention_mask):
52
  if attention_mask[:, -1].sum() == attention_mask.shape[0]: # left padding
@@ -70,24 +83,6 @@ print((query @ docs.T).tolist())
70
  # -> [[0.74, 0.06]]
71
  ```
72
 
73
- ### Input format
74
-
75
- The two sides are asymmetric — encode them the way the model was trained or
76
- retrieval quality drops:
77
-
78
- ```python
79
- QUERY_INSTRUCTION = (
80
- "Instruct: Given a task description, retrieve the most relevant "
81
- "skill document that would help an agent complete the task\nQuery:"
82
- )
83
-
84
- def doc(name, description, body):
85
- return f"{name} | {description} | {body}"
86
- ```
87
-
88
- Queries carry the instruction prefix; skill documents are the bare
89
- `name | description | body` concatenation, no prefix.
90
-
91
  ## Intended use
92
 
93
  First-stage retrieval over a large skill registry: encode the registry offline,
 
36
 
37
  ## Usage
38
 
39
+ The two sides are encoded asymmetrically a task description carries an
40
+ instruction prefix, a skill is the bare `name | description | body`
41
+ concatenation. Encode them the way the model was trained or retrieval quality
42
+ drops. Embeddings come back L2-normalized, so cosine similarity is a plain dot
43
+ product.
44
 
45
  ```python
46
  import torch
 
51
  tok = AutoTokenizer.from_pretrained(MODEL, padding_side="left")
52
  model = AutoModel.from_pretrained(MODEL, dtype=torch.bfloat16).cuda().eval()
53
 
54
+ QUERY_INSTRUCTION = (
55
+ "Instruct: Given a task description, retrieve the most relevant "
56
+ "skill document that would help an agent complete the task\nQuery:"
57
+ )
58
+
59
+
60
+ def doc(name, description, body):
61
+ return f"{name} | {description} | {body}"
62
+
63
 
64
  def last_token_pool(hidden, attention_mask):
65
  if attention_mask[:, -1].sum() == attention_mask.shape[0]: # left padding
 
83
  # -> [[0.74, 0.06]]
84
  ```
85
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
86
  ## Intended use
87
 
88
  First-stage retrieval over a large skill registry: encode the registry offline,