inference/README.md

AREX-Base Inference

This folder provides a minimal one-turn inference example and the complete BrowseComp prompts. It follows the XML tool-call protocol used by the public AREX evaluation code.

Serve the model

Run the following commands from the model repository root. Recent versions of vLLM, SGLang, or another OpenAI-compatible server with Qwen3.5 support can be used. For a text-only vLLM deployment:

vllm serve . \
  --served-model-name AREX-Base \
  --tensor-parallel-size 8 \
  --max-model-len 262144 \
  --reasoning-parser qwen3 \
  --language-model-only

The example uses eight-way tensor parallelism as a starting point. Adjust the parallelism and maximum context length for your hardware.

Run one generation

Install the client:

pip install -U openai

Then send a BrowseComp-style question:

export AREX_BASE_URL="http://127.0.0.1:8000/v1"
export AREX_API_KEY="EMPTY"
export AREX_MODEL="AREX-Base"

python inference/inference.py \
  --question "Your BrowseComp question"

The script returns the model's next action. When it emits an XML <tool_call>, execute that tool, append the assistant output to the message history, and add the real tool result as:

<tool_response>
actual tool result
</tool_response>

Continue until the model calls finish. The example intentionally leaves tool execution to the caller.

Use the prompts directly

prompts.py exports the BrowseComp system and user prompt constants. Tool descriptions are already embedded in the system prompt, so only the question needs formatting:

from inference.prompts import (
    BROWSECOMP_SYSTEM_PROMPT,
    BROWSECOMP_USER_PROMPT,
)

question = "Your BrowseComp question"
messages = [
    {"role": "system", "content": BROWSECOMP_SYSTEM_PROMPT},
    {
        "role": "user",
        "content": BROWSECOMP_USER_PROMPT.format(question=question),
    },
]

build_messages(question) is a convenience wrapper for the same formatting.

BrowseComp exposes search, google_scholar, visit, update_context, and finish.