README.md

---
license: llama3.3
base_model: meta-llama/Llama-3.3-70B-Instruct
library_name: peft
pipeline_tag: text-generation
tags:
  - agentic
  - tool-calling
  - function-calling
  - lora
  - qlora
  - unsloth
  - elicitation
  - rag
language:
  - en
model-index:
  - name: astraforge-70b-TCR
    results:
      - task:
          type: text-generation
          name: Agentic tool-calling
        dataset:
          type: in-house-agentic-benchmark
          name: 79Labs in-house agentic benchmark (N=100)
        metrics:
          - type: accuracy
            name: Tool-correct (right tool, schema-valid call, ≤2 turns)
            value: 0.81
          - type: accuracy
            name: Confirmed-first (asks before acting)
            value: 0.94
          - type: accuracy
            name: GSM8K (reasoning control, exact-match)
            value: 0.93
---

# astraforge-70b-TCR — Tool-Calling & Retrieval Agent (LoRA on Llama-3.3-70B)

**Developed by [79Labs](https://huggingface.co/79Labs) · Version 1.0.0 · [Changelog](./CHANGELOG.md)**

`astraforge-70b-TCR` (Tool-Calling / Retrieval) is a LoRA adapter for
**`meta-llama/Llama-3.3-70B-Instruct`** specialised for **reliable agentic tool use**: retrieving the
right tool from a large catalog (RAG), **eliciting missing parameters**, **confirming before acting**,
emitting schema-valid calls, and staying grounded — while **preserving the base model's reasoning**. It
is *not* a general capability upgrade; it is a focused, measurable improvement on the agentic behaviours
that make a tool-using assistant trustworthy in production.

- **Base:** Llama-3.3-70B-Instruct (4-bit QLoRA)
- **Adapter:** LoRA (r=16), ~828 MB
- **Training:** continued SFT on **1,006,113** agentic examples; eval-gated early stop (best val loss ≈ 0.1198)
- **Serving:** load the adapter with PEFT / Unsloth on the base model (see *Usage*)

> **What the evidence supports (and what it doesn't).** On an in-house agentic benchmark this model
> **matches the base on reasoning** (GSM8K) while **substantially improving tool-calling correctness**
> and learning a **confirm-before-call discipline the base lacks**. The full results table, run log, and
> machine-readable scores are included in [`benchmarks/`](./benchmarks) so every number here is
> verifiable. Sample size is stated (N=100); treat small-N differences as indicative, not leaderboard-grade.

---

## What it does

Trained across **35 business domains** (sales/CRM, finance, support, logistics, healthcare, HR, IT, …)
on these agentic behaviours:

- **Tool calling** in OpenAI function-call JSON — the right tool, schema-valid arguments.
- **Elicitation** — when required parameters are missing, it *asks* instead of hallucinating them.
- **Confirm-before-call** — for consequential actions it asks the user to confirm, then calls.
- **Multi-document RAG** — pick the relevant document, ground the answer, cite it.
- **ReAct / planning** — reason → act → observe chains; plan before acting.
- **Guardrails** — never call an undeclared tool; refuse when out of scope / unsure.
- **Analytics reasoning** — arithmetic + stats (median / variance / %-change / forecast) over supplied data.

## Intended use & scope

**Use it for** building tool-using / function-calling agents where reliability of the *agentic protocol*
(right tool, ask-then-act, confirm, don't hallucinate tools) matters — especially private / on-prem
deployments where a hosted frontier API isn't an option.

**Do not expect** frontier general intelligence. This is a 70B open model specialised on a narrow skill
set. For open-ended coding or research, use a larger / code-specialised model.

---

## Evaluation

Comparable open models under **identical settings**: 4K context (`max_seq_len=4096`, also AstraForge's
trained/served window), greedy decoding, 2048-token budget so reasoning models finish, answer extracted
after any `</think>` and from `\boxed{}` where present, **N = 100**. Each model is prompted in **its own
native tool format** (its tokenizer chat template) so none is penalised for a foreign format.

| Model | Reasoning (GSM8K) | Tool-correct | Confirmed-first |
|---|---|---|---|
| **astraforge-70b-TCR (this model)** | **0.93** | **0.81** | **0.94** |
| Llama-3.3-70B-Instruct (base) | 0.93 | 0.54 | 0.00 |
| Qwen3-32B | 0.80 | 0.79 | 0.02 |
| gpt-oss-120b | 0.87 | 0.43 | 0.05 |
| gpt-oss-20b | 0.87 | 0.46 | 0.06 |
| Gemma-4-31B-it | — | — | — *(excluded: generation hang under Unsloth; did not complete)* |

**Δ vs base:** reasoning **+0.00**, tool-correct **+0.27**, confirmed-first **+0.94**.
Evidence: [`benchmarks/nway_results.json`](./benchmarks/nway_results.json),
[`benchmarks/nway_run.log`](./benchmarks/nway_run.log),
methodology in [`benchmarks/BENCHMARK_METHODOLOGY.md`](./benchmarks/BENCHMARK_METHODOLOGY.md).

**Reading it honestly:**
- **Confirmed-first favours this model by design — that is the point, not a trick.** The other models
  were never trained/prompted to confirm before acting, so their score is near zero. The claim is
  *"this model learned a confirm-before-call protocol,"* **not** *"other models are bad at agents."* If
  your application doesn't want a confirmation step, weight this metric accordingly.
- **Tool-correct is the fairer head-to-head:** whether the model ultimately emits a valid call to the
  right tool. The +0.27 vs base reflects genuine specialisation.
- **Reasoning is a control, not a headline** — the goal was *no regression*; GSM8K parity shows the
  finetune didn't lobotomise general ability.
- **N = 100 is indicative, not leaderboard-grade.** Treat a few points as noise.

> Additional standardized external benchmarks are in progress and will be added in a future revision once
> validated end-to-end.

---

## Usage

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "meta-llama/Llama-3.3-70B-Instruct"
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto", load_in_4bit=True)
model = PeftModel.from_pretrained(model, "79Labs/astraforge-70b-TCR")
tok = AutoTokenizer.from_pretrained("79Labs/astraforge-70b-TCR")

tools = [{
    "type": "function",
    "function": {
        "name": "book_flight",
        "description": "Book a flight for a traveler.",
        "parameters": {"type": "object",
            "properties": {"traveler_name": {"type": "string"}, "origin": {"type": "string"},
                           "destination": {"type": "string"}, "depart_date": {"type": "string"}},
            "required": ["traveler_name", "origin", "destination", "depart_date"]}}}]
msgs = [{"role": "user", "content": "Book Ada a flight from SFO to JFK on 2026-08-01."}]
ids = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(input_ids=ids, max_new_tokens=256)[0][ids.shape[1]:], skip_special_tokens=True))
```

> With Unsloth: `FastLanguageModel.from_pretrained("79Labs/astraforge-70b-TCR", load_in_4bit=True)`.

---

## Training

- **Data:** 1,006,113 deduplicated agentic examples, **self-verified by construction** (every positive
  passes an oracle; hard negatives fail it) across 35 domains + tool catalogs, continuous
  elicitation → confirmation → call, multi-doc RAG, ReAct, plans, guardrails, and real + synthetic
  reasoning. *(Training data is not distributed with this repo.)*
- **Method:** continued SFT (warm-started from a prior SFT champion), 4-bit QLoRA (r=16),
  `max_seq_len=4096`, LR `1e-5`, effective batch 16, Unsloth gradient checkpointing.
- **Governor:** held-out validation with early stopping (`load_best_model_at_end`) — stopped at
  convergence, best checkpoint at val loss ≈ **0.1198**.
- **Hardware:** single NVIDIA GB10 (Grace-Blackwell, 128 GB unified memory).

## Limitations & risks

- **Context: 4K trained / 128K max.** The base supports 128K, so it *runs* at any context ≤128K, but
  the agentic behaviours were reinforced within ~4K — keep the working context (system prompt +
  retrieved tools + dialogue) within ~4K for best fidelity. Pairs naturally with tool-RAG.
- 70B open model — below frontier on general tasks; specialised, not general-purpose.
- Synthetic eval tools — real deployments must wire real executors and **keep confirm-before-call and
  schema-validation guardrails in the harness**, not rely on the model alone.
- Confirmation / elicitation phrasing is English-centric. Inherits base-model biases.

## License

Governed by the **Llama 3.3 Community License** (inherited from the base model).

## Citation

```bibtex
@misc{astraforge70b_tcr_2026,
  title  = {astraforge-70b-TCR: Tool-Calling and Retrieval Agent (LoRA on Llama-3.3-70B)},
  author = {79Labs},
  year   = {2026},
  note   = {Continued SFT on 1M agentic examples; eval-gated. Benchmark harness + raw evidence included.},
  url    = {https://huggingface.co/79Labs/astraforge-70b-TCR}
}
```