---
license: apache-2.0
library_name: pytorch
tags:
  - language-model
  - causal-lm
  - gpt
  - from-scratch
  - educational
pipeline_tag: text-generation
framework: pytorch
---

# SimBot GPT (Level 1)

SimBot GPT is a **from-scratch GPT-style language model** implemented in **PyTorch**.
This project is focused on **learning LLM internals**, not on instruction tuning or production use.

---

## Model Overview

- **Architecture:** Decoder-only Transformer (GPT-like)
- **Training Objective:** Causal Language Modeling
- **Dataset:** Domain-specific text (Simdega / regional data)
- **Purpose:** Educational (understanding how LLMs work internally)

⚠️ This is a **base language model**, not instruction-tuned and not grounded with RAG.

---

## Repository Contents

- `simbot.safetensors` — model weights (safe & HF-recommended format)
- `tokenizer.json` — BPE tokenizer
- `config.json` — model hyperparameters
- `model/simbot.py` — model architecture (PyTorch)

---

## Requirements (Inference Only)

The following packages are **required to load and run the model**:

```txt
torch==2.9.1
tokenizers==0.22.1
safetensors
```
---

## Usage Example

```python
import json
from safetensors.torch import load_file
from tokenizers import Tokenizer
from model.simbot import SIMGPT

# Load tokenizer
tokenizer = Tokenizer.from_file("tokenizer.json")

# Load config
with open("config.json") as f:
    cfg = json.load(f)

# Build model
model = SIMGPT(
    vocab_size=cfg["vocab_size"],
    block_size=cfg["block_size"],
    n_layers=cfg["n_layers"],
    n_heads=cfg["n_heads"],
    d_model=cfg["d_model"]
)

# Load weights
state_dict = load_file("simbot.safetensors")
model.load_state_dict(state_dict)
model.eval()
```

## Prompting the Model

This model is a custom PyTorch implementation and does not support the Hugging Face inference widget.

### Interactive Usage (Recommended)

```bash
python inference.py