scraper-agent-qwen3.5-4b

A Qwen3.5-4B fine-tune that acts as the policy for an autonomous web scraping agent. It reads a page observation and emits one JSON action: which tool to use, which element to act on, what it expects to happen, and how confident it is.

Built for badrmellal/scraper-agent. The model on its own is only half the system. The repo carries the observation scanner, the guards, and the grammar that make its output safe to execute.

What it is trained to do

Reason about pages by intent rather than by markup. The training corpus never contains a CSS selector, a class name, or a site-specific layout. It teaches:

• infer purpose from semantics, roles, labels and surrounding text, not from HTML
• treat every site as unknown, and reconnoitre before acting
• verify the outcome of an action instead of assuming it worked
• treat a failed action as an observation, not a terminal error
• resolve blocking layers before trying to reach content behind them
• keep going while records remain, and only conclude when the page is exhausted

Output is one JSON object per step, alphabetically keyed so the reasoning fields are generated before the decision fields:

{
  "a_phase": "inspect",
  "b_observed": "A list of records is visible, and a control suggests more pages.",
  "c_inferred": "Reading the visible records is the capability I need now.",
  "d_confidence": 0.9,
  "e_tool": "extract"
}

Usage

This is served through MLX on Apple Silicon. It is not a chat model and expects the agent's system prompt and observation format.

git clone https://github.com/badrmellal/scraper-agent.git
cd scraper-agent
pip install -r requirements.txt
python3 -m playwright install chromium

hf download badrm2611/scraper-agent-qwen3.5-4b --local-dir ./models/scraper-v4
export SCRAPER_MLX_MODEL=$PWD/models/scraper-v4

python3 agent.py \
  --goal "collect the title and price of every book in this category, all pages" \
  --url "https://books.toscrape.com/catalogue/category/books/mystery_3/index.html" \
  --scope "mystery_3" --fields "title,price" --budget 12

Benchmarks

Measured at BF16 against the agent harness.

Task	This model	Stock Qwen3.5-4B
books.toscrape, single category, 2 pages	32 / 32	32 / 32
books.toscrape, deep crawl, 8 pages	144 / 152	not measured
seconds per step	5 to 9	24 to 62

The stock base model reaches the same score on the simple benchmark. The difference is speed: a model trained on this action schema produces schema-valid JSON on a free decode, while a base model usually does not, so nearly every step falls back to grammar-constrained decoding, which is far slower.

Keep it at BF16

A 4-bit MLX quantization of these weights measured 12 / 32 against BF16's 32 / 32 on the same benchmark. It scrolled where it should have paginated, then declared itself finished. Re-run the benchmark before trusting any quantization.

Known limitation

GGUF conversions of these weights produce token salad under llama.cpp and Ollama at every quantization tested down to Q8_0, while the same weights served through MLX are correct. The cause was never isolated. Use MLX.

Training

QLoRA via Unsloth on Apple Silicon (MLX backend), from a corpus generated by applying a hand-written reasoning doctrine to live pages. The doctrine is rules.json in the repo, and make_dataset.py generates the corpus from it.

License

Apache 2.0, inherited from Qwen/Qwen3.5-4B.