---
library_name: fields-lm
license: apache-2.0
pipeline_tag: text-generation
tags:
- pytorch
- causal-lm
- long-context
- state-space-model
- mamba
- research
language:
- en
datasets:
- emozilla/pg19
---

# Fields 300M PG-19

This repository contains the first public checkpoint for **Fields LM**, an experimental causal language-model architecture combining native Field recurrence, displaced readout, PCAF successor memory, local refresh attention, and Mamba-2 editors.

## Model details

- architecture: Fields 18F/2M/4R + PCAF;
- scale: approximately 300M parameters;
- vocabulary: 16,384-token BPE;
- training data: controlled sample of PG-19;
- training budget: 49,152,000 tokens;
- training context: 2,048 tokens;
- evaluated contexts: 2K, 8K, 16K, 32K, and 64K;
- weights: `safetensors`;
- runtime: install from the official GitHub repository.

This is **not** a frontier-scale assistant and is not instruction-tuned. Long-context values beyond 2K measure evaluation-time extrapolation, not training at those lengths.

## Installation

```bash
pip install "git+https://github.com/Multisymboliccore/fields-lm.git"
```

Install the validated CUDA dependencies for Mamba-2 and causal convolution as documented in the GitHub repository.

## Loading

```python
import torch
from fields_official import FieldsHubModel

model = FieldsHubModel.from_pretrained(
    "Multisymboliccore/fields-300m-pg19",
    map_location="cpu",
)
model = model.to(device="cuda", dtype=torch.bfloat16).eval()
```

## Intended use

The checkpoint is intended for architecture research, reproducibility, controlled ablations, long-context analysis, and further pretraining or fine-tuning by qualified users.

## Limitations

- trained with a comparatively small token budget;
- English literary-book domain bias from PG-19;
- not instruction-tuned or safety-tuned;
- may generate inaccurate, biased, or harmful text;
- custom CUDA dependencies are required for the validated high-performance path;
- 64K evaluation does not imply equal quality across all long-context tasks.

## Evaluation

Final three-seed metrics and paper links will be synchronized from the validated release artifacts. Do not infer universal model quality from the architecture benchmark alone.

## License

Apache License 2.0. Dependency licenses continue to apply.