README.md

---
license: apache-2.0
language:
- en
library_name: transformers
tags:
- text-generation
- causal-lm
- transformer
- argonne
- instruct
- sft
- dpo
pipeline_tag: text-generation
---

# Argonne 3.0-instruct

Argonne 3.0-instruct is a 2.88B-parameter instruction-tuned language model from the Argonne 3.x family. It is the SFT+DPO finetuned version of [Argonne 3.0-base](https://huggingface.co/PursuitOfDataScience/argonne-3.0-base), trained on UltraChat (SFT) and KatoHF Chatbot Arena (DPO) datasets.

The base model was pretrained on ~76B tokens of FineWeb text at 1,024 context length. The instruct variant extends context to 13,568 tokens via RoPE extrapolation (θ = 1,000,000) and is trained for instruction following, dialogue, and multi-turn conversation.

## Model architecture

| Component | Specification |
|-----------|---------------|
| **Parameters** | 2,882,162,688 (~2.88B) |
| **Layers** | 24 transformer blocks |
| **Hidden size** | 3,072 |
| **Attention heads** | 12 query / 4 key-value (GQA) |
| **Head dimension** | 256 |
| **Feed-forward** | SwiGLU MLP, 8,192 intermediate dim |
| **Attention pattern** | Interleaved local/global causal attention |
| **Local attention window** | 256 tokens (every other layer) |
| **Normalization** | RMSNorm with QK / V / sandwich norms |
| **Position encoding** | RoPE (θ = 1,000,000) |
| **Logit stabilization** | Final logit softcap = 15.0 |
| **Context length** | 13,568 tokens (RoPE extrapolated from 1,024-ctx base) |
| **Vocabulary size** | 151,669 |
| **Tied embeddings** | Yes (input ↔ output) |

## Training details

### Stage 1 — Supervised Fine-Tuning (SFT)

| Item | Value |
|------|-------|
| **Script** | `sft.py` |
| **Dataset** | [HuggingFaceH4/ultrachat_200k](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) |
| **Dataset recipe** | `sft_ultrachat` (system + user/assistant turns) |
| **Context length** | 13,568 tokens |
| **Batch size per GPU** | 10 |
| **Gradient accumulation** | 2 |
| **Effective batch** | 271,360 tokens/step |
| **Optimizer** | AdamW (β₁=0.9, β₂=0.95, weight decay 0.1) |
| **Peak learning rate** | 2.0e-5 |
| **Min LR ratio** | 0.1 |
| **Schedule** | Warmup-Stable-Decay; 200 warmup steps |
| **Total optimizer steps** | 10,500 |
| **Epochs** | 1 |
| **Checkpoint cadence** | 30 minutes (time-based, `save_total_limit=4`) |
| **Hardware** | 1× NVIDIA H200 GPU |
| **Random seed** | 42 |

### Stage 2 — Direct Preference Optimization (DPO)

| Item | Value |
|------|-------|
| **Script** | `dpo.py` |
| **Dataset** | [KatoHF/chatbot_arena_binarized](https://huggingface.co/datasets/KatoHF/chatbot_arena_binarized) |
| **Dataset recipe** | `chat_refine_strict` |
| **Context length** | 13,568 tokens |
| **Batch size per GPU** | 4 |
| **Gradient accumulation** | 2 |
| **Optimizer** | AdamW |
| **Peak learning rate** | 1.0e-6 |
| **Beta (DPO temperature)** | 0.03 |
| **Score mode** | `avg` |
| **Checkpoint cadence** | 30 minutes (time-based, `save_total_limit=4`) |
| **Hardware** | 1× NVIDIA H200 GPU |
| **Random seed** | 42 |

## Training data

| Item | Value |
|------|-------|
| **SFT corpus** | UltraChat 200k — multi-turn instruction-response pairs; see [HuggingFaceH4/ultrachat_200k](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k) |
| **DPO corpus** | KatoHF Chatbot Arena — binarized preference pairs from real user comparisons; see [KatoHF/chatbot_arena_binarized](https://huggingface.co/datasets/KatoHF/chatbot_arena_binarized) |
| **Tokenizer** | [Qwen/Qwen3-0.6B-Base](https://huggingface.co/Qwen/Qwen3-0.6B-Base) (151,669-token vocab), reused from the base model |

## Tokenizer

This model reuses the Qwen3 tokenizer (vocabulary size 151,669) through the `Qwen2Tokenizer` compatibility class. The tokenizer files are bundled with the checkpoint so no extra download is required.

## Source code

Built from the GitHub main branch: https://github.com/PursuitOfDataScience/ArgonneAI/tree/main

Key scripts used to produce this checkpoint:
- [`model.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/model.py) — the `ArgonneCausalLM` / `ArgonneConfig` architecture (bundled here as `model.py`)
- [`sft.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/sft.py) — supervised fine-tuning loop
- [`dpo.py`](https://github.com/PursuitOfDataScience/ArgonneAI/blob/main/dpo.py) — DPO preference optimization loop

## Inference

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "PursuitOfDataScience/argonne-3.0-instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
)

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
model.eval()

messages = [
    {"role": "user", "content": "Explain what a black hole is in a way a 10-year-old would understand."}
]
prompt_ids = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
)
input_ids = torch.tensor([prompt_ids], dtype=torch.long, device=device)

seed = 444
torch.manual_seed(seed)
if device.startswith("cuda"):
    torch.cuda.manual_seed_all(seed)

output_ids = model.generate(
    input_ids,
    max_length=input_ids.shape[1] + 200,
    temperature=0.8,
    top_p=0.9,
    do_sample=True,
    repetition_penalty=1.3,
    no_repeat_ngram_size=4,
)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
```

## Recommended inference settings

| Parameter | Value |
|-----------|-------|
| **Context length** | 13,568 tokens |
| **Temperature** | 0.8 |
| **Top-p** | 0.9 |
| **Repetition penalty** | 1.3 |
| **No-repeat n-gram size** | 4 |
| **Seed** | 444 |
| **Continuation length** | 200 new tokens |

## Usage notes

- Load with `trust_remote_code=True` so the custom `ArgonneCausalLM` / `ArgonneConfig` classes (`model.py`) are registered.
- Use `apply_chat_template()` for instruction prompts; the model ships with a Jinja2 chat template in `tokenizer_config.json`.
- The custom `generate` method on `ArgonneCausalLM` uses `max_length` (total sequence length) rather than `max_new_tokens`; see the snippet above for the recommended pattern.
- Weights are published as bf16 safetensor shards with a `model.safetensors.index.json` weight map for sharded loading.
- The published context length is 13,568 tokens (RoPE extrapolated from the 1,024-ctx base).

## Limitations

- 2.88B parameters — significantly smaller than frontier models; expect weaker performance on complex reasoning, math, and code tasks.
- Context length extended via RoPE extrapolation; long-context performance may degrade on tasks requiring precise retrieval beyond the original 1,024-ctx pretraining distribution.
- SFT trained on UltraChat (English-only, curated conversation data); limited multilingual capability.
- DPO trained on Chatbot Arena preference data; alignment quality depends on the preference dataset coverage.
- No safety filtering or content moderation has been applied.

## Citation

```bibtex
@misc{argonne30instruct,
  author = {PursuitOfDataScience},
  title = {Argonne 3.0-instruct},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/PursuitOfDataScience/argonne-3.0-instruct}
}
```