quanfire-dev commited on
Commit
b4b9404
·
verified ·
1 Parent(s): 595ee34

Correct the usage snippet: load_adapter takes a local directory and returns (encoder, metadata); encode_batch for lists

Browse files
Files changed (1) hide show
  1. README.md +28 -9
README.md CHANGED
@@ -67,26 +67,45 @@ and no difference between them may be read as the cost of anything.
67
 
68
  ## Using it
69
 
 
 
 
 
 
 
 
70
  ```python
71
  from multilingual_embedding.embedding.neural.adapter import load_adapter
72
 
73
- encoder = load_adapter("quanfire-ai/embed-eulaw-multi")
74
- vectors = encoder.encode(["Article 5 of Regulation (EU) 2016/679 ..."])
 
 
 
 
 
 
75
  ```
76
 
77
- Install with `pip install multilingual-embedding`.
 
 
 
 
 
78
 
79
  ### Base revision
80
 
81
- This adapter's manifest predates our base pinning convention, so it reads back as unpinned.
82
- The base revision it was built against is
83
  `614241f622f53c4eeff9890bdc4f31cfecc418b3`, which we establish from the training host's model
84
- cache rather than from a field the run recorded: that snapshot is the only one present and it
85
- predates the run by a month. Pin it explicitly if you need byte-stable behaviour:
 
86
 
87
  ```python
88
- encoder = load_adapter(
89
- "quanfire-ai/embed-eulaw-multi",
90
  revision="614241f622f53c4eeff9890bdc4f31cfecc418b3",
91
  )
92
  ```
 
67
 
68
  ## Using it
69
 
70
+ Install the package, then download the adapter and load it from the local directory:
71
+
72
+ ```bash
73
+ pip install 'quanfire-multilingual-embedding[neural]'
74
+ hf download quanfire-ai/embed-eulaw-multi --revision v1.0.0 --local-dir embed-eulaw-multi
75
+ ```
76
+
77
  ```python
78
  from multilingual_embedding.embedding.neural.adapter import load_adapter
79
 
80
+ encoder, meta = load_adapter("embed-eulaw-multi")
81
+
82
+ texts = [
83
+ "The processing of personal data shall be lawful only if the data subject has given consent.",
84
+ "Die Verarbeitung personenbezogener Daten ist nur rechtmaessig, wenn die betroffene Person ihre Einwilligung erteilt hat.",
85
+ "Member States shall ensure that fishing vessels exceeding 12 metres carry a satellite tracking device.",
86
+ ]
87
+ vectors = encoder.encode_batch(texts)
88
  ```
89
 
90
+ `load_adapter` returns the encoder together with its metadata, because the metadata carries
91
+ the prefixes needed to use the model correctly. `encode_batch` takes a list of strings;
92
+ `encode` takes a single string.
93
+
94
+ On the three sentences above, the English and German expressions of the same provision sit at
95
+ cosine 0.835, while the unrelated fisheries article sits at 0.379.
96
 
97
  ### Base revision
98
 
99
+ This adapter's manifest predates our base pinning convention, so `meta.checkpoint_revision`
100
+ reads back as `None`. The base revision it was built against is
101
  `614241f622f53c4eeff9890bdc4f31cfecc418b3`, which we establish from the training host's model
102
+ cache rather than from a field the run recorded: that snapshot is the only one present there
103
+ and it predates the run by a month. Pass it explicitly if you need the base held still, since
104
+ an upstream repository can change what sits behind a name:
105
 
106
  ```python
107
+ encoder, meta = load_adapter(
108
+ "embed-eulaw-multi",
109
  revision="614241f622f53c4eeff9890bdc4f31cfecc418b3",
110
  )
111
  ```