README.md

metadata

language: en
license: mit
base_model: BikoRiko/GPT-2.4-High-Pro
tags:
  - gpt2
  - math
  - fine-tuned
  - Pro
  - Math
pipeline_tag: text-generation

GPT-2.5-Math

GPT-2.5-Math is an upgraded version of BikoRiko/GPT-2.4-High-Pro, featuring an expanded architecture and specialized fine-tuning on mathematical reasoning.

Model Details

• Architecture: GPT-2 with 6 additional layers (Total parameters ~0.2B).
• Training Hardware: NVIDIA H100 (via Modal.com).
• Dataset: 5% subset of microsoft/orca-math-word-problems-200k.
• Objective: Fine-tuned to solve math word problems and logical queries.

Performance

The model is trained for mathematical reasoning. While it is a 0.2B parameter model, it demonstrates the beginning of logical grounding for basic word problems.

Training Details

• Optimizer: AdamW
• Precision: Mixed Precision (torch.amp)
• Epochs: 3
• Learning Rate: 5e-5