Multisymboliccore/fields-300m-pg19

Fields 300M PG-19

This repository contains the first public checkpoint for Fields LM, an experimental causal language-model architecture combining native Field recurrence, displaced readout, PCAF successor memory, local refresh attention, and Mamba-2 editors.

Model details

• architecture: Fields 18F/2M/4R + PCAF;
• scale: approximately 300M parameters;
• vocabulary: 16,384-token BPE;
• training data: controlled sample of PG-19;
• training budget: 49,152,000 tokens;
• training context: 2,048 tokens;
• evaluated contexts: 2K, 8K, 16K, 32K, and 64K;
• weights: safetensors;
• runtime: install from the official GitHub repository.

This is not a frontier-scale assistant and is not instruction-tuned. Long-context values beyond 2K measure evaluation-time extrapolation, not training at those lengths.

Installation

pip install "git+https://github.com/Multisymboliccore/fields-lm.git"

Install the validated CUDA dependencies for Mamba-2 and causal convolution as documented in the GitHub repository.

Loading

import torch
from fields_official import FieldsHubModel

model = FieldsHubModel.from_pretrained(
    "Multisymboliccore/fields-300m-pg19",
    map_location="cpu",
)
model = model.to(device="cuda", dtype=torch.bfloat16).eval()

Intended use

The checkpoint is intended for architecture research, reproducibility, controlled ablations, long-context analysis, and further pretraining or fine-tuning by qualified users.

Limitations

• trained with a comparatively small token budget;
• English literary-book domain bias from PG-19;
• not instruction-tuned or safety-tuned;
• may generate inaccurate, biased, or harmful text;
• custom CUDA dependencies are required for the validated high-performance path;
• 64K evaluation does not imply equal quality across all long-context tasks.

Evaluation

Final three-seed metrics and paper links will be synchronized from the validated release artifacts. Do not infer universal model quality from the architecture benchmark alone.

License

Apache License 2.0. Dependency licenses continue to apply.