AuroraGPT-700M-Tools

A 707M-parameter on-device chat model that keeps its normal conversational behaviour and calls a web_search tool — trained with a LoRA on the AuroraGPT-700M SFT, entirely on a laptop CPU, overnight, for $0.

The point of this model: add tool-calling to a small model without wrecking it. A LoRA freezes the base, so the chat behaviour of the underlying SFT is preserved — you get the same model you liked, plus the ability to reach for the web.

What it does

• Chats normally — same behaviour as the AuroraGPT-700M SFT base.
• Calls tools — when a question needs current/live/uncertain info, it emits a Hermes-style tool call as plain text:

  <tool_call>
  {"name": "web_search", "arguments": {"query": "Tokyo weather today"}}
  </tool_call>
  

• Generalises — although it was trained only on web_search, it learned the tool-call format, so in tool-rich apps it will also reach for other declared tools (calculator, fetch_url, etc.). It reliably produces correct arguments for web_search; other tools are hit-or-miss (see limitations).

Prompt / tool format

Native AuroraGPT format (special tokens: <|system|>=2, <|user|>=3, <|assistant|>=4, <|end|>=5, bos <|endoftext|>=0):

<|system|>{system + tool declaration}<|end|><|user|>{question}<|end|><|assistant|>

Tool results are fed back as a user turn wrapping <tool_response>...</tool_response>, and the model then answers from them. The bundled chat_template.jinja handles all of this (injects tools when present, renders <tool_call> / <tool_response>).

How to run it

• LlamaWeb / apps that parse tool calls from text (recommended): the app reads the <tool_call> from the output and runs the search itself. Rock-solid.
• Off Grid: works — enable only the web_search tool in settings. With many tools enabled the model over-eagerly calls all of them, and it only knows the right arguments for web_search.
• LM Playground: chats fine; web search is intermittent — llama.cpp can't match aurora's custom tokens to a known tool format, so it auto-generates a fragile grammar that sometimes crashes ("empty grammar stack"). Prefer a text-parsing app.

Files

• model.safetensors (+ config/tokenizer) — the merged fp32 model (🤗 transformers)
• auroragpt-700m-sft-tools-Q8_0.gguf — Q8_0 quant (~753 MB) for phones / llama.cpp
• adapter_model.safetensors — the standalone LoRA adapter (apply over AuroraGPT-700M)
• chat_template.jinja — tool-aware chat template

Training

• Method: LoRA (r=16, α=32, on q/k/v/o/gate/up/down) over the frozen AuroraGPT-700M SFT — 11.5M trainable params (1.6%). LoRA keeps the base behaviour intact.
• Data: ~3,900 examples — 2,400 non-<think> web_search tool calls (single + two-turn with <tool_response>), 500 restraint ("don't search for 2+2"), 1,000 SmolTalk rehearsal.
• Compute: 1 epoch on an Intel i5-1335U CPU (fp32 base, gradient checkpointing, ~11h), loss ~0.5. No GPU, no cloud, no cost.

Honest limitations

• It's 700M. It fumbles multi-digit arithmetic on its own — which is exactly why a calculator tool helps (delegating fixes its weak spot).
• Tool triggering is inconsistent — it reliably calls web_search for many prompts but not all; a follow-up LoRA with more/varied tool examples would tighten this.
• Arguments are only reliable for web_search — the format generalises to other tools, the argument shapes do not.

Built by SmallAICreator. Trained on a laptop.