| --- |
| license: apache-2.0 |
| language: |
| - en |
| - multilingual |
| base_model: zeroentropy/zembed-1-embedding |
| pipeline_tag: feature-extraction |
| library_name: sentence-transformers |
| tags: |
| - text-embeddings-inference |
| - infinity |
| model_max_length: 32768 |
| --- |
| |
| # zembed-1 for Infinity |
|
|
| Infinity configuration adaptation of [ZeroEntropy's zembed-1-embedding](https://huggingface.co/zeroentropy/zembed-1-embedding), pinned to revision `cf13c81f3274394053d166740294f7eea4586f7a`. Model weights, tokenizer, pooling, and learned projection weights are unchanged. The original model card is retained in [UPSTREAM_README.md](UPSTREAM_README.md), and the original license is retained in [LICENSE](LICENSE). |
|
|
| `modules.json` replaces `modeling_zembed.ZembedTransformer` with the standard Sentence Transformers `Transformer` module so Infinity v3's LibTorch loader accepts the checkpoint. The original wrapper appends `<|im_end|>\n` to each input. **Clients must still append this suffix**; this copy does not make raw-text requests automatically equivalent to the original model. |
|
|
| ## Run |
|
|
| ```bash |
| docker run --rm --gpus all -p 7997:80 -v infinity-hf-cache:/data \ |
| ghcr.io/michaelfeil/infinity_v3:nvidia-3.0.0-alpha.3 \ |
| --model-id michaelfeil/zembed-1-infinity \ |
| --backend libtorch --torch-device cuda --device-id 0 \ |
| --dtype bfloat16 --pooling last-token \ |
| --default-prompt-name document --max-batch-tokens 32768 |
| ``` |
|
|
| The saved `query` and `document` prompts provide the prefix. Supply the suffix exactly once, after the unformatted input text: |
|
|
| ```bash |
| curl http://localhost:7997/embed \ |
| -H 'Content-Type: application/json' \ |
| -d '{"model":"michaelfeil/zembed-1-infinity","inputs":["What is backpropagation?<|im_end|>\n"],"prompt_name":"query","normalize":true,"truncate":false}' |
| ``` |
|
|
| Use `prompt_name: "document"` for documents. [infinity_client.py](infinity_client.py) adds the suffix automatically: |
|
|
| ```python |
| from infinity_client import embed |
| |
| queries = embed(["What is backpropagation?"], kind="query") |
| documents = embed(["Backpropagation computes gradients."], kind="document") |
| ``` |
|
|
| This setup returns normalized, full-width **2560-dimensional** embeddings with last-token pooling. The original learned dimensionality projections are preserved, but are not applied by this Infinity setup; simple truncation is not equivalent to those projections. Inputs above 32768 tokens should be chunked. Avoid truncating away the terminal suffix. |
|
|
| This repository is a configuration adaptation, not a new training run. Weight hashes are recorded in [infinity-preparation.json](infinity-preparation.json). It has not yet undergone an end-to-end embedding parity run against the original Sentence Transformers model. |
|
|
