Blame for README.md

---
license: apache-2.0
language:
- en
- multilingual
base_model: zeroentropy/zembed-1-embedding
pipeline_tag: feature-extraction
library_name: sentence-transformers
tags:
- text-embeddings-inference
- infinity
model_max_length: 32768
---

# zembed-1 for Infinity

Infinity configuration adaptation of [ZeroEntropy's zembed-1-embedding](https://huggingface.co/zeroentropy/zembed-1-embedding), pinned to revision `cf13c81f3274394053d166740294f7eea4586f7a`. Model weights, tokenizer, pooling, and learned projection weights are unchanged. The original model card is retained in [UPSTREAM_README.md](UPSTREAM_README.md), and the original license is retained in [LICENSE](LICENSE).

`modules.json` replaces `modeling_zembed.ZembedTransformer` with the standard Sentence Transformers `Transformer` module so Infinity v3's LibTorch loader accepts the checkpoint. The original wrapper appends `<|im_end|>\n` to each input. **Clients must still append this suffix**; this copy does not make raw-text requests automatically equivalent to the original model.

## Run

```bash
docker run --rm --gpus all -p 7997:80 -v infinity-hf-cache:/data \
  ghcr.io/michaelfeil/infinity_v3:nvidia-3.0.0-alpha.3 \
  --model-id michaelfeil/zembed-1-infinity \
  --backend libtorch --torch-device cuda --device-id 0 \
  --dtype bfloat16 --pooling last-token \
  --default-prompt-name document --max-batch-tokens 32768
```

The saved `query` and `document` prompts provide the prefix. Supply the suffix exactly once, after the unformatted input text:

```bash
curl http://localhost:7997/embed \
  -H 'Content-Type: application/json' \
  -d '{"model":"michaelfeil/zembed-1-infinity","inputs":["What is backpropagation?<|im_end|>\n"],"prompt_name":"query","normalize":true,"truncate":false}'
```

Use `prompt_name: "document"` for documents. [infinity_client.py](infinity_client.py) adds the suffix automatically:

```python
from infinity_client import embed

queries = embed(["What is backpropagation?"], kind="query")
documents = embed(["Backpropagation computes gradients."], kind="document")
```

This setup returns normalized, full-width **2560-dimensional** embeddings with last-token pooling. The original learned dimensionality projections are preserved, but are not applied by this Infinity setup; simple truncation is not equivalent to those projections. Inputs above 32768 tokens should be chunked. Avoid truncating away the terminal suffix.

This repository is a configuration adaptation, not a new training run. Weight hashes are recorded in [infinity-preparation.json](infinity-preparation.json). It has not yet undergone an end-to-end embedding parity run against the original Sentence Transformers model.