Instructions to use Octen/Octen-WebGR-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Octen/Octen-WebGR-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Octen/Octen-WebGR-4B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Octen/Octen-WebGR-4B")
model = AutoModelForCausalLM.from_pretrained("Octen/Octen-WebGR-4B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Octen/Octen-WebGR-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Octen/Octen-WebGR-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Octen/Octen-WebGR-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Octen/Octen-WebGR-4B

SGLang

How to use Octen/Octen-WebGR-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Octen/Octen-WebGR-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Octen/Octen-WebGR-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Octen/Octen-WebGR-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Octen/Octen-WebGR-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Octen/Octen-WebGR-4B with Docker Model Runner:
```
docker model run hf.co/Octen/Octen-WebGR-4B
```

Octen-WebGR-4B / README.md

bflhc

Add arXiv link, contact, Discord and Octen links to the model card

575dcce verified about 19 hours ago

preview code

Raw History Blame Contribute Delete

7.65 kB

---
license: cc-by-nc-4.0
license_link: LICENSE
base_model: Qwen/Qwen3-4B-Base
library_name: transformers
tags:
  - generative-retrieval
  - semantic-id
  - navigational-search
  - web-search
---
# Octen-WebGR-4B

[![Discord](https://img.shields.io/badge/Discord-Join%20Octen-5865F2?logo=discord&logoColor=white)](https://discord.gg/Y3Yq5qxk7h)

Octen-WebGR-4B is a generative retriever for navigational web search, developed by [Octen](https://octen.ai/). Given a query, it generates a two-token semantic identifier
(SID) that is constrained to a table of 195.9 million real URLs, so the web index lives in the model's parameters: retrieval needs
one GPU and about 10 GB of memory, instead of hundreds of gigabytes of vectors.

It is described in the technical report [*Octen-WebGR: Scaling Generative Retrieval to 200 Million Web Pages*](https://arxiv.org/abs/2610.35256) (Xu et al., 2026).
A smaller model for a 31-million-URL table is available as [Octen-WebGR-0.6B](https://huggingface.co/Octen/Octen-WebGR-0.6B).

## Model

| | |
|---|---|
| Base model | Qwen3-4B-Base, full-parameter fine-tuning |
| Layers / hidden size | 36 / 2,560 |
| Parameters, total (non-embedding) | 4.36B (3.63B) |
| Identifier tokens | 2 × 65,536 (vocabulary ids 151,936 to 283,007) |
| URLs in the training universe | 201,231,792 |
| URLs in the released table | 195,921,090 (after screening, see below) |
| Training data | Query–URL click pairs up to August 2026 (proprietary, not released) |
| Weights | bf16, 8.7 GB |

## Results

Navigational queries from September 2026, the month after the training data, weighted by clicks. A system scores a hit when the
clicked target page is among its top *k* URLs; targets outside a system's candidate set count as misses.

| System | hit@1 | hit@10 |
|---|---|---|
| **Octen-WebGR-4B** | **59.6** | **68.7** |
| Octen-WebGR-0.6B (31M-URL universe) | 56.5 | 64.0 |
| Dense retrieval (Octen embedding + adapter trained on the same clicks, exact search) | 24.8 | 55.6 |
| Dense retrieval + click prior | 46.5 | 64.2 |
| BM25 over URL texts (URL + training queries) | 9.1 | 25.3 |
| Lookup in the training click table | 58.9 | 62.5 |

GR is strongest on new ways of asking for familiar pages: on unseen queries for pages with many training queries it ranks the target
first 30.9% of the time, against 17.2% for dense retrieval (21.4% with the click prior). Dense retrieval remains better on rarely
visited pages, and a click table cannot answer new queries at all. All numbers use the full 201M universe; restricting decoding to the screened table lowers hit@1 from 59.6 to 58.7,
because 1.2% of the traffic targets a removed URL. See the report for the full evaluation.

## How it works

1. Normalise the query: collapse whitespace, strip, lowercase. Keep at most the first 32 tokens.
2. Prompt: `Query: {q}\nURL:` without special tokens.
3. Generate two SID tokens. Level *t* (0 or 1) uses vocabulary id `151936 + t * 65536 + code`.
4. Constrained beam search: at each level, only prefixes present in `sid_table/` are allowed and the softmax is taken over the valid
   children only; 20 beams are kept. The score is the sum of the two log-probabilities.
5. Look `(sid0, sid1)` up in `sid_table/` to get the URL.

> **Do not use `model.generate()`.** The tokenizer has no entries for the SID tokens, and unconstrained generation returns
> identifiers that map to no URL. Use the inference code in this repository.

## Usage

```bash
pip install -r requirements.txt      # torch 2.8.0, transformers 4.57.1; flash-attn optional
python infer_gpu.py --model Octen/Octen-WebGR-4B --query "youtube" "boeing internship"
python infer_gpu.py --model Octen/Octen-WebGR-4B --check        # compare against smoke_test.json
```

```python
from huggingface_hub import snapshot_download
import sys
root = snapshot_download("Octen/Octen-WebGR-4B")
sys.path.insert(0, root)
from infer_gpu import GR
gr = GR(root)                        # loads the model and the 195.9M-row table
for url, logp, row in gr.search(["hugging face"], K=10)[0]:
    print(url, logp)
```

`infer_gpu.py` is a reference implementation: it recomputes the prompt for every beam at the second level and pads each batch to its
longest prompt. On an idle H200 it answers a single query in 56 ms (median) and needs 10.1 ms per query in batches of 32.
`infer_cpu.py` implements the same protocol on CPUs and reuses the prompt's key–value cache.

Memory: about 9.7 GiB of GPU memory (bf16 weights and SID keys). Peak host RAM is about 23 GiB with `infer_cpu.py`, which keeps the weights and the table in memory.

## Files

| Path | Content | License |
|---|---|---|
| `*.safetensors`, `config.json`, `generation_config.json` | Model weights (bf16) | CC BY-NC 4.0 |
| `tokenizer*.json`, `vocab.json`, `merges.txt` | Qwen3 tokenizer, unchanged | Apache 2.0 |
| `sid_table/part-*.parquet` | One row per URL: `sid0`, `sid1` (int32), `url` (string, without scheme), sorted by `sid0 * 65536 + sid1`, unique | CC BY-NC 4.0 |
| `infer_gpu.py`, `infer_cpu.py` | Inference code | Apache 2.0 |
| `smoke_test.json` | 31 hand-picked queries with expected top-20 URLs, for `--check` | – |

## Screening of the table

Before release we removed every URL whose host, or a parent domain of it, is on the adult, malware, phishing, cryptojacking or
stalkerware lists of the UT1 blacklists; every URL listed by URLhaus; and every URL whose text contains an e-mail address or a
credential (5,310,702 URLs, 2.6%, mostly adult sites). Removing a row from the table removes the URL from every possible output, because
decoding is constrained to the table. The tables were not checked against lists of child sexual abuse material, nor screened for
personal pages; both are left to the takedown process.

URL removal requests: research@octen.ai. Removals are published as versioned changes to `sid_table/`.

## Intended use and limitations

- Intended for research on generative retrieval and for navigational search over the URLs in the released table.
- The model can only return URLs in the table. Pages that appeared after August 2026 are not covered, and adding a page requires
  training on it.
- It is weaker than dense retrieval on rarely visited pages, and it is not meant for informational queries.
- Its knowledge reflects clicks up to August 2026 and the biases of the rankings under which those clicks were made.
- It was trained on aggregated click data without user identifiers. It must not be used to infer anything about individual users.

## License

The model weights and the SID-to-URL table are released under [CC BY-NC 4.0](LICENSE) (non-commercial use). The inference code is
released under the [Apache License 2.0](LICENSE-CODE). The model is derived from Qwen3-4B-Base, Copyright 2024 Alibaba Cloud, licensed
under the Apache License 2.0 ([LICENSE-QWEN3](LICENSE-QWEN3)); see [NOTICE](NOTICE). Unlike the Octen-Embedding models, the WebGR
weights are not licensed for commercial use; contact Octen for commercial licensing.

## Citation

```bibtex
@misc{octen2026webgr,
  title         = {Octen-WebGR: Scaling Generative Retrieval to 200 Million Web Pages},
  author        = {Xu, Guangwei and Zhang, Wenkai and Liu, Yawen and Zhang, Dongxu and Xu, Xijie and Zou, Kuan},
  year          = {2026},
  eprint        = {2610.35256},
  archivePrefix = {arXiv},
  primaryClass  = {cs.IR},
  url           = {https://arxiv.org/abs/2610.35256}
}
```

## Contact

For API access, deployment solutions, and technical documentation, visit [octen.ai](https://octen.ai/).

Join our [Discord community](https://discord.gg/Y3Yq5qxk7h) for questions, feedback, and the latest updates.

	---
	license: cc-by-nc-4.0
	license_link: LICENSE
	base_model: Qwen/Qwen3-4B-Base
	library_name: transformers
	tags:
	- generative-retrieval
	- semantic-id
	- navigational-search
	- web-search
	---

	# Octen-WebGR-4B

	[![Discord](https://img.shields.io/badge/Discord-Join%20Octen-5865F2?logo=discord&logoColor=white)](https://discord.gg/Y3Yq5qxk7h)

	Octen-WebGR-4B is a generative retriever for navigational web search, developed by [Octen](https://octen.ai/). Given a query, it generates a two-token semantic identifier
	(SID) that is constrained to a table of 195.9 million real URLs, so the web index lives in the model's parameters: retrieval needs
	one GPU and about 10 GB of memory, instead of hundreds of gigabytes of vectors.

	It is described in the technical report [Octen-WebGR: Scaling Generative Retrieval to 200 Million Web Pages](https://arxiv.org/abs/2610.35256) (Xu et al., 2026).
	A smaller model for a 31-million-URL table is available as [Octen-WebGR-0.6B](https://huggingface.co/Octen/Octen-WebGR-0.6B).

	## Model

	\| \| \|
	\|---\|---\|
	\| Base model \| Qwen3-4B-Base, full-parameter fine-tuning \|
	\| Layers / hidden size \| 36 / 2,560 \|
	\| Parameters, total (non-embedding) \| 4.36B (3.63B) \|
	\| Identifier tokens \| 2 × 65,536 (vocabulary ids 151,936 to 283,007) \|
	\| URLs in the training universe \| 201,231,792 \|
	\| URLs in the released table \| 195,921,090 (after screening, see below) \|
	\| Training data \| Query–URL click pairs up to August 2026 (proprietary, not released) \|
	\| Weights \| bf16, 8.7 GB \|

	## Results

	Navigational queries from September 2026, the month after the training data, weighted by clicks. A system scores a hit when the
	clicked target page is among its top k URLs; targets outside a system's candidate set count as misses.

	\| System \| hit@1 \| hit@10 \|
	\|---\|---\|---\|
	\| Octen-WebGR-4B \| 59.6 \| 68.7 \|
	\| Octen-WebGR-0.6B (31M-URL universe) \| 56.5 \| 64.0 \|
	\| Dense retrieval (Octen embedding + adapter trained on the same clicks, exact search) \| 24.8 \| 55.6 \|
	\| Dense retrieval + click prior \| 46.5 \| 64.2 \|
	\| BM25 over URL texts (URL + training queries) \| 9.1 \| 25.3 \|
	\| Lookup in the training click table \| 58.9 \| 62.5 \|

	GR is strongest on new ways of asking for familiar pages: on unseen queries for pages with many training queries it ranks the target
	first 30.9% of the time, against 17.2% for dense retrieval (21.4% with the click prior). Dense retrieval remains better on rarely
	visited pages, and a click table cannot answer new queries at all. All numbers use the full 201M universe; restricting decoding to the screened table lowers hit@1 from 59.6 to 58.7,
	because 1.2% of the traffic targets a removed URL. See the report for the full evaluation.

	## How it works

	1. Normalise the query: collapse whitespace, strip, lowercase. Keep at most the first 32 tokens.
	2. Prompt: `Query: {q}\nURL:` without special tokens.
	3. Generate two SID tokens. Level t (0 or 1) uses vocabulary id `151936 + t * 65536 + code`.
	4. Constrained beam search: at each level, only prefixes present in `sid_table/` are allowed and the softmax is taken over the valid
	children only; 20 beams are kept. The score is the sum of the two log-probabilities.
	5. Look `(sid0, sid1)` up in `sid_table/` to get the URL.

	> Do not use `model.generate()`. The tokenizer has no entries for the SID tokens, and unconstrained generation returns
	> identifiers that map to no URL. Use the inference code in this repository.

	## Usage

	```bash
	pip install -r requirements.txt # torch 2.8.0, transformers 4.57.1; flash-attn optional
	python infer_gpu.py --model Octen/Octen-WebGR-4B --query "youtube" "boeing internship"
	python infer_gpu.py --model Octen/Octen-WebGR-4B --check # compare against smoke_test.json
	```

	```python
	from huggingface_hub import snapshot_download
	import sys

	root = snapshot_download("Octen/Octen-WebGR-4B")
	sys.path.insert(0, root)
	from infer_gpu import GR

	gr = GR(root) # loads the model and the 195.9M-row table
	for url, logp, row in gr.search(["hugging face"], K=10)[0]:
	print(url, logp)
	```

	`infer_gpu.py` is a reference implementation: it recomputes the prompt for every beam at the second level and pads each batch to its
	longest prompt. On an idle H200 it answers a single query in 56 ms (median) and needs 10.1 ms per query in batches of 32.
	`infer_cpu.py` implements the same protocol on CPUs and reuses the prompt's key–value cache.

	Memory: about 9.7 GiB of GPU memory (bf16 weights and SID keys). Peak host RAM is about 23 GiB with `infer_cpu.py`, which keeps the weights and the table in memory.

	## Files

	\| Path \| Content \| License \|
	\|---\|---\|---\|
	\| `*.safetensors`, `config.json`, `generation_config.json` \| Model weights (bf16) \| CC BY-NC 4.0 \|
	\| `tokenizer*.json`, `vocab.json`, `merges.txt` \| Qwen3 tokenizer, unchanged \| Apache 2.0 \|
	\| `sid_table/part-.parquet` \| One row per URL: `sid0`, `sid1` (int32), `url` (string, without scheme), sorted by `sid0 65536 + sid1`, unique \| CC BY-NC 4.0 \|
	\| `infer_gpu.py`, `infer_cpu.py` \| Inference code \| Apache 2.0 \|
	\| `smoke_test.json` \| 31 hand-picked queries with expected top-20 URLs, for `--check` \| – \|

	## Screening of the table

	Before release we removed every URL whose host, or a parent domain of it, is on the adult, malware, phishing, cryptojacking or
	stalkerware lists of the UT1 blacklists; every URL listed by URLhaus; and every URL whose text contains an e-mail address or a
	credential (5,310,702 URLs, 2.6%, mostly adult sites). Removing a row from the table removes the URL from every possible output, because
	decoding is constrained to the table. The tables were not checked against lists of child sexual abuse material, nor screened for
	personal pages; both are left to the takedown process.

	URL removal requests: research@octen.ai. Removals are published as versioned changes to `sid_table/`.

	## Intended use and limitations

	- Intended for research on generative retrieval and for navigational search over the URLs in the released table.
	- The model can only return URLs in the table. Pages that appeared after August 2026 are not covered, and adding a page requires
	training on it.
	- It is weaker than dense retrieval on rarely visited pages, and it is not meant for informational queries.
	- Its knowledge reflects clicks up to August 2026 and the biases of the rankings under which those clicks were made.
	- It was trained on aggregated click data without user identifiers. It must not be used to infer anything about individual users.

	## License

	The model weights and the SID-to-URL table are released under [CC BY-NC 4.0](LICENSE) (non-commercial use). The inference code is
	released under the [Apache License 2.0](LICENSE-CODE). The model is derived from Qwen3-4B-Base, Copyright 2024 Alibaba Cloud, licensed
	under the Apache License 2.0 ([LICENSE-QWEN3](LICENSE-QWEN3)); see [NOTICE](NOTICE). Unlike the Octen-Embedding models, the WebGR
	weights are not licensed for commercial use; contact Octen for commercial licensing.

	## Citation

	```bibtex
	@misc{octen2026webgr,
	title = {Octen-WebGR: Scaling Generative Retrieval to 200 Million Web Pages},
	author = {Xu, Guangwei and Zhang, Wenkai and Liu, Yawen and Zhang, Dongxu and Xu, Xijie and Zou, Kuan},
	year = {2026},
	eprint = {2610.35256},
	archivePrefix = {arXiv},
	primaryClass = {cs.IR},
	url = {https://arxiv.org/abs/2610.35256}
	}
	```

	## Contact

	For API access, deployment solutions, and technical documentation, visit [octen.ai](https://octen.ai/).

	Join our [Discord community](https://discord.gg/Y3Yq5qxk7h) for questions, feedback, and the latest updates.