Create README.md

2f394c5 verified 1 day ago

7.09 kB

license: apache-2.0
language:
  - en
  - es
  - fr
  - de
  - it
  - pt
  - ru
  - ar
  - hi
  - ko
  - zh
library_name: transformers
base_model:
  - arcee-ai/Trinity-Large-Preview
base_model_relation: quantized

Trinity-Large-Preview-FP8-Block

Introduction

Trinity-Large-Preview is a 398B-parameter sparse Mixture-of-Experts (MoE) model with approximately 13B active parameters per token. It is the largest model in Arcee AI's Trinity family, trained on more than 17 trillion tokens and delivering frontier-level performance with strong long-context comprehension. Trinity-Large-Preview is a lightly post-trained model based on Trinity-Large-Base.

This repository contains the FP8 block-quantized weights of Trinity-Large-Preview (FP8 weights and activations with per-block scaling).

Try it at chat.arcee.ai

More details on the training of Trinity Large are available in the technical report.

Quantization Details

Scheme: FP8 Block (FP8 weights and activations, per-block scaling with E8M0 scale format)
Format: compressed-tensors
Intended use: High-throughput FP8 deployment of Trinity-Large-Preview with near-lossless quality, optimized for NVIDIA Hopper/Blackwell GPUs
Supported backends: DeepGEMM, vLLM CUTLASS, Triton

Model Variants

The Trinity Large family consists of three checkpoints from the same training run:

Trinity-Large-Preview: Lightly post-trained, chat-ready model undergoing active RL
Trinity-Large-TrueBase: 10T-token pre-anneal pretraining checkpoint
Trinity-Large-Base: Full 17T-token pretrained foundation model with mid-training anneals

Architecture

Trinity-Large-Preview uses a sparse MoE configuration designed to maximize efficiency while maintaining large-scale capacity.

Hyperparameter	Value
Total parameters	~398B
Active parameters per token	~13B
Experts	256 (1 shared)
Active experts	4
Routing strategy	4-of-256 (1.56% sparsity)
Dense layers	6
Pretraining context length	8,192
Context length after extension	512k
Architecture	Sparse MoE (AfmoeForCausalLM)

Benchmarks

Benchmark	Llama 4 Maverick	Trinity-Large Preview
MMLU	85.5	87.2
MMLU-Pro	80.5	75.2
GPQA-Diamond	69.8	63.3
AIME 2025	19.3	24.0

Training Configuration

Pretraining

Training tokens: 17 trillion
Data partner: Datology

Posttraining

This checkpoint was instruction tuned on 20B tokens.

Infrastructure

Hardware: 2,048 NVIDIA B300 GPUs
Parallelism: HSDP + Expert Parallelism
Compute partner: Prime Intellect

Usage

Running our model

Inference tested on

8x NVIDIA H100 80GB (tensor parallel = 8)
vLLM 0.18.0+

VLLM

Supported in VLLM release 0.18.0+ with DeepGEMM FP8 MoE acceleration.

# pip
pip install "vllm>=0.18.0"

Serving the model with DeepGEMM enabled:

VLLM_USE_DEEP_GEMM=1 vllm serve arcee-ai/Trinity-Large-Preview-FP8-Block \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

Serving without DeepGEMM (falls back to CUTLASS/Triton):

vllm serve arcee-ai/Trinity-Large-Preview-FP8-Block \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

Transformers

Use the main transformers branch or pass trust_remote_code=True with a released version.

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "arcee-ai/Trinity-Large-Preview-FP8-Block"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    trust_remote_code=True
)

messages = [
    {"role": "user", "content": "Who are you?"},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    input_ids,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.8,
    top_k=50,
    top_p=0.8
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

API

Available on OpenRouter:

curl -X POST "https://openrouter.ai/v1/chat/completions" \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "arcee-ai/trinity-large-preview",
    "messages": [
      {
        "role": "user",
        "content": "What are some fun things to do in New York?"
      }
    ]
  }'

License

Trinity-Large-Preview-FP8-Block is released under the Apache License, Version 2.0.

Citation

If you use this model, please cite:

@misc{singh2026arceetrinity,
  title        = {Arcee Trinity Large Technical Report},
  author       = {Varun Singh and Lucas Krauss and Sami Jaghouar and Matej Sirovatka and Charles Goddard and Fares Obied and Jack Min Ong and Jannik Straube and Fern and Aria Harley and Conner Stewart and Colin Kealty and Maziyar Panahi and Simon Kirsten and Anushka Deshpande and Anneketh Vij and Arthur Bresnu and Pranav Veldurthi and Raghav Ravishankar and Hardik Bishnoi and DatologyAI Team and Arcee AI Team and Prime Intellect Team and Mark McQuade and Johannes Hagemann and Lucas Atkins},
  year         = {2026},
  eprint       = {2602.17004},
  archivePrefix= {arXiv},
  primaryClass = {cs.LG},
  doi          = {10.48550/arXiv.2602.17004},
  url          = {https://arxiv.org/abs/2602.17004}
}