---
license: mit
tags:
  - pytorch
  - gpt2
  - instruction-tuning
  - sft
  - slm
  - from-scratch
  - raschka
base_model: nishantup/nanogpt-slm-124m
---

# GPT2 SLM Instruct (Raschka Architecture) -- 163.2M Parameters

Instruction fine-tuned Small Language Model using the Raschka-style GPTModel architecture.

**Pipeline:** Trained from scratch -> Pretrained on 133 classic English fiction books -> SFT on Alpaca-format instructions.

## Quick Start

### Option 1: Run directly (downloads model + runs examples)
```bash
pip install torch tiktoken huggingface_hub
python gpt2_slm_instruct_inference.py
```

### Option 2: Import and use `ask()` in your own code
```python
# Import loads the model automatically (one-time download from HuggingFace)
from gpt2_slm_instruct_inference import ask

# Simple question
print(ask("What is the capital of France?"))
print()

# With input context
print(ask(
    instruction="Summarize the following text.",
    input_text="Machine learning enables systems to learn from data rather than being explicitly programmed."
))
print()

# Control generation
print(ask(
    "Write a short poem about the ocean.",
    temperature=1.0,    # higher = more creative
    top_k=100,          # wider sampling pool
    max_tokens=150      # longer output
))
print()
```

### Option 3: Load weights manually
```python
from huggingface_hub import hf_hub_download
import torch

model_path = hf_hub_download(
    repo_id="nishantup/gpt2-slm-instruct",
    filename="gpt2_slm_instruct.pth"
)

from gpt2_slm_instruct_inference import GPTModel, BASE_CONFIG

model = GPTModel(BASE_CONFIG)
model.load_state_dict(torch.load(model_path, map_location="cpu"))
model.eval()
```

## Prompt Format

```
Below is an instruction that describes a task.

### Instruction:
{instruction}

### Response:
```

With optional input:
```
Below is an instruction that describes a task, paired with further context.

### Instruction:
{instruction}

### Input:
{input}

### Response:
```

## Model Details

| Attribute | Value |
|:---|:---|
| Parameters | 163.2M |
| Architecture | Raschka GPTModel (12 layers, 12 heads, 768 dim) |
| Context length | 256 tokens |
| Tokenizer | tiktoken GPT-2 BPE (50,257 tokens) |
| Base model | [nishantup/nanogpt-slm-124m](https://huggingface.co/nishantup/nanogpt-slm-124m) (`gpt_slm_best.pth`) |
| Fine-tuning | Supervised (Alpaca format, 1,100 examples, 2 epochs) |
| Framework | PyTorch |

## Architecture Comparison

| Feature | This model (Raschka) | nanoGPT variant |
|:---|:---|:---|
| Weights file | `gpt2_slm_instruct.pth` | `nanogpt_slm_instruct.pth` |
| Attention | Separate W_query, W_key, W_value | Combined c_attn |
| LayerNorm | scale/shift params | weight/bias params |
| MLP | FeedForward (Sequential) | MLP (c_fc/c_proj) |
| Config | Dict (BASE_CONFIG) | Dataclass (GPTConfig) |
| Weight tying | No | Yes (wte = lm_head) |
| forward() returns | logits | (logits, loss) tuple |

## Files

| File | Description |
|:---|:---|
| `gpt2_slm_instruct.pth` | SFT fine-tuned weights (Raschka GPTModel) |
| `gpt2_slm_instruct_inference.py` | Standalone inference script -- import and call `ask()` |
| `config.json` | Model configuration |

## `ask()` API Reference

```python
ask(instruction, input_text="", max_tokens=256, temperature=0.7, top_k=40)
```

| Parameter | Default | Description |
|:---|:---|:---|
| `instruction` | (required) | The task instruction |
| `input_text` | `""` | Optional additional context |
| `max_tokens` | `256` | Maximum tokens to generate |
| `temperature` | `0.7` | 0.0 = greedy, 0.7 = balanced, 1.5 = creative |
| `top_k` | `40` | Top-k filtering (None = no filtering) |

## Related Models

| Variant | Architecture | Repo |
|:---|:---|:---|
| Pretrained base (Raschka) | GPTModel | [nishantup/nanogpt-slm-124m](https://huggingface.co/nishantup/nanogpt-slm-124m) (`gpt_slm_best.pth`) |
| Pretrained base (nanoGPT) | GPT | [nishantup/nanogpt-slm-124m](https://huggingface.co/nishantup/nanogpt-slm-124m) (`nanogpt_slm_best.pth`) |
| Instruct SFT (nanoGPT) | GPT | [nishantup/nanogpt-slm-instruct](https://huggingface.co/nishantup/nanogpt-slm-instruct) |