Remove TEI snippet temporarily
Browse files
README.md
CHANGED
|
@@ -90,56 +90,6 @@ embeddings = model.encode(texts, quantization="binary") # Shape: (5, 2560), quan
|
|
| 90 |
|
| 91 |
</details>
|
| 92 |
|
| 93 |
-
<details>
|
| 94 |
-
<summary>Using Text Embeddings Inference (TEI)</summary>
|
| 95 |
-
|
| 96 |
-
> [!NOTE]
|
| 97 |
-
> Text Embeddings Inference v1.9.0 will be released stable soon, in the meantime
|
| 98 |
-
> feel free to use the latest containers or rather via SHA ``.
|
| 99 |
-
|
| 100 |
-
> [!IMPORTANT]
|
| 101 |
-
> Currently, only int8-quantized embeddings are available via TEI. Remember to use cosine similarity with unnormalized int8 embeddings.
|
| 102 |
-
|
| 103 |
-
- CPU w/ Candle:
|
| 104 |
-
|
| 105 |
-
```bash
|
| 106 |
-
docker run -p 8080:80 ghcr.io/huggingface/text-embeddings-inference:cpu-latest --model-id perplexity-ai/pplx-embed-v1-4B --auto-truncate
|
| 107 |
-
```
|
| 108 |
-
|
| 109 |
-
- CPU w/ ORT (ONNX Runtime):
|
| 110 |
-
|
| 111 |
-
```bash
|
| 112 |
-
docker run -p 8080:80 ghcr.io/huggingface/text-embeddings-inference:cpu-latest --model-id onnx-community/pplx-embed-v1-4B --auto-truncate
|
| 113 |
-
```
|
| 114 |
-
|
| 115 |
-
- GPU w/ CUDA:
|
| 116 |
-
|
| 117 |
-
```bash
|
| 118 |
-
docker run --gpus all --shm-size 1g -p 8080:80 ghcr.io/huggingface/text-embeddings-inference:cuda-latest --model-id perplexity-ai/pplx-embed-v1-4B --auto-truncate
|
| 119 |
-
```
|
| 120 |
-
|
| 121 |
-
> Alternatively, when running in CUDA you can use the architecture / compute capability specific
|
| 122 |
-
> container instead of the `cuda-latest`, as that includes the binaries for Turing, Ampere and
|
| 123 |
-
> Hopper, so using a dedicated container will be lighter e.g., `ampere-latest`.
|
| 124 |
-
|
| 125 |
-
And then you can send requests to it via cURL to `/embed`:
|
| 126 |
-
|
| 127 |
-
```bash
|
| 128 |
-
curl http://0.0.0.0:8080/embed \
|
| 129 |
-
-H "Content-Type: application/json" \
|
| 130 |
-
-d '{
|
| 131 |
-
"inputs": [
|
| 132 |
-
"Scientists explore the universe driven by curiosity.",
|
| 133 |
-
"Children learn through curious exploration.",
|
| 134 |
-
"Historical discoveries began with curious questions.",
|
| 135 |
-
"Animals use curiosity to adapt and survive.",
|
| 136 |
-
"Philosophy examines the nature of curiosity."
|
| 137 |
-
],
|
| 138 |
-
"normalize": false
|
| 139 |
-
}'
|
| 140 |
-
```
|
| 141 |
-
|
| 142 |
-
</details>
|
| 143 |
<details>
|
| 144 |
<summary> Using ONNX models </summary>
|
| 145 |
|
|
|
|
| 90 |
|
| 91 |
</details>
|
| 92 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 93 |
<details>
|
| 94 |
<summary> Using ONNX models </summary>
|
| 95 |
|