metadata
license: apache-2.0
language:
- en
tags:
- text-generation
- education
- student-handbook
- campus-qa
- custom-architecture
pipeline_tag: text-generation
CampGPT-Student-Handbook
A compact GPT model trained for university student handbook Q&A.
Model Details
| Property | Value |
|---|---|
| Parameters | 322,680,576 (322.7M) |
| Architecture | Transformer (GQA + RoPE + SwiGLU + MoE) |
| Layers | 12 |
| Heads | 12 (KV: 4) |
| Embedding | 768 |
| Context Length | 1024 |
| Tokenizer | tiktoken (GPT-2, 50257 vocab) |
| Training | Pretrain -> SFT -> DPO |
Training Pipeline
- Pretrain: 10B tokens from FineWeb-Edu
- SFT: Fine-tuned on student handbook Q&A pairs
- DPO: Preference optimization with chosen/rejected pairs
Usage
from serve import CampGPTServer
server = CampGPTServer("campgpt-student-handbook")
response = server.chat("What are the requirements for a scholarship?")
print(response)
Chat Format
### System:
You are a helpful university assistant...
### User:
What are the scholarship requirements?
### Assistant:
Based on the student handbook...
Limitations
- Small model with limited capacity
- Knowledge limited to the specific student handbook used for training
- May hallucinate details not in the training data