---
license: apache-2.0
base_model: fitleech/lm-2b
tags:
- text-generation
- chronogpt
- sn38
- cutoff-2013
- pytorch
---

# lm-2b-typemix-2013

A 2B-parameter ChronoGPT causal language model fine-tuned from
[`fitleech/lm-2b`](https://huggingface.co/fitleech/lm-2b) for one epoch on
[`fitleech/sn38-type-mix-2013-500m`](https://huggingface.co/datasets/fitleech/sn38-type-mix-2013-500m).

The training corpus uses a 2013 knowledge cutoff and a 20% / 50% / 30%
Type1 / Type2 / Type3 mixture. Weights are the completed `epoch-1` checkpoint.

## Architecture

- 52 layers
- 12 attention heads
- 1536 model dimension
- 50,304-token GPT-2-compatible vocabulary
- BF16 checkpoint in safetensors format