Instructions to use Octen/Octen-WebGR-0.6B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Octen/Octen-WebGR-0.6B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Octen/Octen-WebGR-0.6B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Octen/Octen-WebGR-0.6B")
model = AutoModelForCausalLM.from_pretrained("Octen/Octen-WebGR-0.6B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Octen/Octen-WebGR-0.6B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Octen/Octen-WebGR-0.6B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Octen/Octen-WebGR-0.6B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Octen/Octen-WebGR-0.6B

SGLang

How to use Octen/Octen-WebGR-0.6B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Octen/Octen-WebGR-0.6B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Octen/Octen-WebGR-0.6B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Octen/Octen-WebGR-0.6B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Octen/Octen-WebGR-0.6B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Octen/Octen-WebGR-0.6B with Docker Model Runner:
```
docker model run hf.co/Octen/Octen-WebGR-0.6B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Octen-WebGR-0.6B

Octen-WebGR-0.6B is a small generative retriever for navigational web search, developed by Octen. Given a query, it generates a two-token semantic identifier (SID) that is constrained to a table of 30.9 million real URLs, so the web index lives in the model's parameters. It needs 1.6 GiB of GPU memory and also runs on CPUs (72 ms per query on one server socket).

It is described in the technical report Octen-WebGR: Scaling Generative Retrieval to 200 Million Web Pages (Xu et al., 2026). The larger model for a 196-million-URL table is Octen-WebGR-4B. The 31M universe of this model is a subset of the 4B model's universe and uses the same identifiers.

Model


Base model	Qwen3-0.6B-Base, full-parameter fine-tuning
Layers / hidden size	28 / 1,024
Parameters, total (non-embedding)	0.73B (0.44B)
Identifier tokens	2 × 65,536 (vocabulary ids 151,936 to 283,007)
URLs in the training universe	31,282,165
URLs in the released table	30,931,888 (after screening, see below)
Training data	Query–URL click pairs up to August 2026 (proprietary, not released)
Weights	bf16, 1.5 GB

Results

Navigational queries from September 2026, the month after the training data, weighted by clicks. A system scores a hit when the clicked target page is among its top k URLs; targets outside a system's candidate set count as misses, so this model, which only returns URLs from its 31M universe, is scored against the same targets as the systems over the 201M universe.

System	hit@1	hit@10
Octen-WebGR-4B (201M-URL universe)	59.6	68.7
Octen-WebGR-0.6B	56.5	64.0
Dense retrieval (Octen embedding + adapter trained on the same clicks, exact search, 201M)	24.8	55.6
Dense retrieval + click prior	46.5	64.2
BM25 over URL texts (URL + training queries)	9.1	25.3
Lookup in the training click table	58.9	62.5

The 0.6B model is within 3.1 points of the 4B model at hit@1 and matches it on new queries for familiar pages, but it finds few rarely visited pages, most of which are outside its universe. All numbers use the full 31M universe; restricting decoding to the screened table lowers hit@1 from 56.5 to 55.7. See the report for the full evaluation.

How it works

Normalise the query: collapse whitespace, strip, lowercase. Keep at most the first 32 tokens.
Prompt: Query: {q}\nURL: without special tokens.
Generate two SID tokens. Level t (0 or 1) uses vocabulary id 151936 + t * 65536 + code.
Constrained beam search: at each level, only prefixes present in sid_table/ are allowed and the softmax is taken over the valid children only; 20 beams are kept. The score is the sum of the two log-probabilities.
Look (sid0, sid1) up in sid_table/ to get the URL.

Do not use model.generate(). The tokenizer has no entries for the SID tokens, and unconstrained generation returns identifiers that map to no URL. Use the inference code in this repository.

Usage

pip install -r requirements.txt      # torch 2.8.0, transformers 4.57.1; flash-attn optional
python infer_cpu.py --model Octen/Octen-WebGR-0.6B --query "youtube" "boeing internship" --threads 16
python infer_gpu.py --model Octen/Octen-WebGR-0.6B --query "youtube"
python infer_cpu.py --model Octen/Octen-WebGR-0.6B --check --threads 16    # compare against smoke_test.json

from huggingface_hub import snapshot_download
import sys

root = snapshot_download("Octen/Octen-WebGR-0.6B")
sys.path.insert(0, root)
from infer_cpu import GRCPU

gr = GRCPU(root)                     # loads the model and the 30.9M-row table
for url, logp, row in gr.search(["hugging face"], K=10)[0]:
    print(url, logp)

infer_cpu.py reuses the prompt's key–value cache for the second level and takes the softmax over valid children only. On one socket of an Intel Xeon 8568Y+ (bf16, cores pinned) it answers a single query in 72 ms (median) and about 57 queries per second in batches of 8; its hit@1 is within 0.1 points of the GPU implementation. infer_gpu.py is a reference implementation that recomputes the prompt for every beam; on an idle H200 it answers a single query in 43.5 ms and needs 3.9 ms per query in batches of 32.

Memory: 1.6 GiB of GPU memory. Peak host RAM is about 4.5 GiB with infer_cpu.py, which keeps the weights and the table in memory.

Files

Path	Content	License
`model.safetensors`, `config.json`, `generation_config.json`	Model weights (bf16)	CC BY-NC 4.0
`tokenizer*.json`, `vocab.json`, `merges.txt`	Qwen3 tokenizer, unchanged	Apache 2.0
`sid_table/part-*.parquet`	One row per URL: `sid0`, `sid1` (int32), `url` (string, without scheme), sorted by `sid0 * 65536 + sid1`, unique	CC BY-NC 4.0
`infer_gpu.py`, `infer_cpu.py`	Inference code	Apache 2.0
`smoke_test.json`	31 hand-picked queries with expected top-20 URLs, for `--check`	–

Screening of the table

Before release we removed every URL whose host, or a parent domain of it, is on the adult, malware, phishing, cryptojacking or stalkerware lists of the UT1 blacklists; every URL listed by URLhaus; and every URL whose text contains an e-mail address or a credential (350,277 URLs, 1.1%). Removing a row from the table removes the URL from every possible output, because decoding is constrained to the table. The tables were not checked against lists of child sexual abuse material, nor screened for personal pages; both are left to the takedown process.

URL removal requests: research@octen.ai. Removals are published as versioned changes to sid_table/.

Intended use and limitations

Intended for research on generative retrieval and for navigational search over the URLs in the released table.
The model can only return URLs in the table. Pages that appeared after August 2026 are not covered, and adding a page requires training on it.
It is weaker than dense retrieval on rarely visited pages, and it is not meant for informational queries.
Its knowledge reflects clicks up to August 2026 and the biases of the rankings under which those clicks were made.
It was trained on aggregated click data without user identifiers. It must not be used to infer anything about individual users.

License

The model weights and the SID-to-URL table are released under CC BY-NC 4.0 (non-commercial use). The inference code is released under the Apache License 2.0. The model is derived from Qwen3-0.6B-Base, Copyright 2024 Alibaba Cloud, licensed under the Apache License 2.0 (LICENSE-QWEN3); see NOTICE. Unlike the Octen-Embedding models, the WebGR weights are not licensed for commercial use; contact Octen for commercial licensing.

Citation

@misc{octen2026webgr,
  title         = {Octen-WebGR: Scaling Generative Retrieval to 200 Million Web Pages},
  author        = {Xu, Guangwei and Zhang, Wenkai and Liu, Yawen and Zhang, Dongxu and Xu, Xijie and Zou, Kuan},
  year          = {2026},
  eprint        = {2610.35256},
  archivePrefix = {arXiv},
  primaryClass  = {cs.IR},
  url           = {https://arxiv.org/abs/2610.35256}
}