README.md

---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
  - sports
  - sports-analytics
  - structured-generation
  - json
  - gpt2
  - from-scratch
  - prototype
language:
  - en
---

# StatPredict Lite

A small (~33M parameter) GPT trained **from scratch** to read an athlete's previous
statistical seasons and generate a predicted next season as **structured JSON**.

No pretrained weights were used. The architecture is GPT-2, but every parameter here was
randomly initialized and trained only on college athletics season records.

## Why "Lite"

**"Lite" means this is an open source prototype.** It is the public, fully inspectable
version of the idea, released so people can read the recipe, run the model, and build on it.

The whole thing is small on purpose. A compact parameter count, a compact vocabulary, and a
tokenizer built specifically for numeric structured text mean it trains on a single GPU for
pennies and runs comfortably on a laptop CPU. That makes it easy to fork, retrain on your
own sport or your own stat schema, and experiment with quickly.

## What it does

The model is trained on a text format where the input and the target are separated by `>>>`:

```
{input json} >>> {output json}
```

At inference you supply everything left of the separator, and the model writes the predicted
season on the right.

**Example (fabricated, illustrative only):**

Input:
```json
{"sport":"women's basketball","sex":"female","position":"guard","position_groups":["guard"],
 "previous_seasons":[{"season":"2024-25","stats":{"G":30,"PTS":400,"AST":90,"Tot Reb":150}}]}
```

Output:
```json
{"predicted_next_season":{"season":"2025-26","stats":{"G":31,"PTS":455,"AST":102,"Tot Reb":168}}}
```

## Quick start

```python
import json, torch
from transformers import GPT2LMHeadModel, PreTrainedTokenizerFast

repo = "grichard99/statpredict-lite"
tok = PreTrainedTokenizerFast.from_pretrained(repo)
model = GPT2LMHeadModel.from_pretrained(repo).eval()

athlete = {
    "sport": "women's basketball",
    "sex": "female",
    "position": "guard",
    "position_groups": ["guard"],
    "previous_seasons": [
        {"season": "2024-25", "stats": {"G": 30, "PTS": 400, "AST": 90, "Tot Reb": 150}}
    ],
}

prompt = json.dumps(athlete, separators=(",", ":"), ensure_ascii=False)
ids = [tok.bos_token_id] + tok.encode(prompt, add_special_tokens=False) + [tok.sep_token_id]

with torch.no_grad():
    out = model.generate(
        torch.tensor([ids]),
        max_new_tokens=320,
        do_sample=False,
        pad_token_id=tok.pad_token_id,
        eos_token_id=tok.eos_token_id,
    )

text = tok.decode(out[0][len(ids):], skip_special_tokens=True)
print(json.loads(text))
```

> The `>>>` separator is the tokenizer's `sep_token`. Prepend `bos_token` and append
> `sep_token` exactly as shown. The model was trained with that framing and will behave
> poorly without it.

## Input schema

| Field | Type | Notes |
|---|---|---|
| `sport` | string | One of the supported sports below |
| `sex` | string | `male`, `female`, or `unknown` |
| `position` | string | Sport appropriate position label |
| `position_groups` | list of strings | Broader grouping; may repeat `position` |
| `previous_seasons` | list of objects | Each `{"season": str, "stats": {...}}`, oldest first |

Optional `conference` and `division` fields are also understood.

**Supported sports:** baseball, men's basketball, men's soccer, softball, track & field,
women's basketball, women's soccer, women's volleyball.

Stat keys are sport specific and match the source records. Basketball uses `PTS`, `AST`,
`Tot Reb`, `FG%` and similar; volleyball uses `Kills`, `Digs`, `Aces`; track & field uses
`Shot Put`, `Mile`, `5000 Meters`. Track & field values are in seconds for timed events and
meters for field events. Unrecognized keys are not guaranteed to be handled sensibly.

## How it was built

| | |
|---|---|
| Architecture | GPT-2 (`GPT2LMHeadModel`), randomly initialized |
| Parameters | ~33M |
| Layers / heads / d_model | 10 / 8 / 512 |
| Context length | 1024 |
| Vocabulary | 1,888 (custom BPE) |
| Precision | float32 |

The tokenizer is a custom BPE trained on the corpus itself, with **digits split into
individual tokens** so numbers are represented consistently rather than being absorbed into
arbitrary multi digit merges. Training used early stopping on a held out validation split,
**partitioned by athlete**, so no athlete appears in both train and validation.

Full training recipe, data preparation scripts, and evaluation harness:
**[github.com/grichard99/statpredict-lite](https://github.com/grichard99/statpredict-lite)**

## Training data

Trained on college athletics season stat lines compiled from public results sources.

**The dataset is not published and is not included here.** It is also structurally
de-identified. A training record contains only sport, sex, position, position group,
conference, division, season label, and numeric stat values. It contains **no athlete names,
no school names, and no identifiers**. Those fields never existed in the training
representation, so the model has no capacity to emit them. The published tokenizer
vocabulary reflects this: it contains sport, position, conference, and stat name tokens
only.

Every example shown in this card is fabricated.

## Limitations and intended use

This is a research prototype released for inspection and experimentation. It is not a
validated forecasting system, and it is not fit for any consequential decision. Please do
not use it for recruiting, scouting, evaluation, wagering, or anything else affecting a real
person.

No performance claims are made here. **If you intend to rely on it for anything, evaluate it
yourself on your own held out data first.** The evaluation harness in the GitHub repo,
including a naive "repeat last season" baseline to compare against, is there for exactly
that purpose.

Other things worth knowing:

- Coverage is limited to the eight sports listed above, and to the stat keys and season
  formats present in the source records.
- Output is free form generated text parsed as JSON. It is usually valid, but you should
  wrap parsing in a `try` / `except` and handle failures.
- The model tends to return the full stat schema for a sport, so it may emit stat keys you
  did not supply, or omit ones you did.
- Athletes with unusual, sparse, or single season histories are the weakest case.
- Structural zeros are common in the underlying data. A sprinter has no throwing marks, and
  a field player has no goalkeeping stats. This shapes model behavior in ways that are easy
  to misread when scoring, so score accordingly.
- Greedy decoding (`do_sample=False`) is recommended. Sampling makes malformed output
  considerably more likely.

## License

Apache-2.0, for both the weights and the code.

Because the model was trained from scratch with no pretrained initialization, it carries no
inherited license obligations from any upstream model. The license above is the only one
that applies.

## Acknowledgements

Trained on a single **NVIDIA RTX A4000** and evaluated on an **NVIDIA A6000**, both via
[NVIDIA Brev](https://developer.nvidia.com/brev).

## Citation

```bibtex
@software{statpredict_lite,
  title  = {StatPredict Lite: a from-scratch small language model for structured
            athlete season prediction},
  author = {Richard, Guy},
  year   = {2026},
  url    = {https://huggingface.co/grichard99/statpredict-lite}
}
```