---
license: apache-2.0
library_name: transformers
pipeline_tag: text-classification
tags: [laya, minimal, snapshot, english-checkpoint, fine-tuning, french]
---

# Laya — Minimal Snapshot (English Checkpoint)

This repository is a **minimalist snapshot** of the [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) model hub. It keeps **only the English base checkpoint** (`laya`, ModernBERT-large backbone, 421M params) together with the files needed to run and fine-tune it.

**Upstream:**
- Model hub: [convaiinnovations/laya](https://huggingface.co/convaiinnovations/laya)
- Code & docs: [github.com/NandhaKishorM/laya](https://github.com/NandhaKishorM/laya)
- Package: `pip install laya`

## What was removed

- `multilingual/` and `typed-decisions/` checkpoints (~1.4 GB of weights)
- Benchmark images and eval results (`assets/`, `eval/`)
- Legacy research scripts (`rl_agent_api.py`, `email_utils.py`)

The English weights themselves are **byte-identical to the upstream snapshot** — this repo does not modify, retrain, or re-quantize the original model in any way. It is simply a trimmed copy, ideal for pulling the model onto a device with limited disk space (~810 MB total, fully standalone and offline-capable).

The multilingual and typed-decisions checkpoints remain available from the upstream hub if you ever need them.

## Layout

```
config.json                  model marker (architectures: LayaTypedDecisions)
model.safetensors            full weights: encoder + decision heads (~804 MB)
rl_agent_config.json         hyperparameters and calibration temperatures
encoder/config.json          ModernBERT-large architecture config
tokenizer/                   tokenizer.json + tokenizer_config.json
rl_common.py                 training reference (RLCD: model build, rewards, metrics)
test.py                      standalone smoke test
```

## Quickstart

```bash
pip install laya
```

The checkpoint is fully self-contained: the `model.safetensors` already embeds the ModernBERT-large encoder, so nothing is downloaded from the Hugging Face hub at load time.

```python
import os
os.environ["HF_HUB_OFFLINE"] = "1"  # run fully offline

import laya

agent = laya.load(".")  # loads this directory
```

## Concrete example (`test.py`)

Non-autoregressive typed decisions over a state, all questions answered in a **single forward pass**:

```python
import os
os.environ["HF_HUB_OFFLINE"] = "1"
import laya
import time

# Load the local checkpoint
agent = laya.load(".")

# State to evaluate
article = ("By 2031, artificial intelligences no longer merely answer: they anticipate, "
           "negotiate and sometimes push back. In newsrooms, hospitals and courtrooms, "
           "autonomous agents propose decisions, draft summaries and flag their own "
           "uncertainty. Some observers speak of a 'second cognitive wave': systems capable "
           "of doubting, asking for clarification and refusing a poorly specified task. "
           "Others worry about a silent dependency, where humans approve without truly "
           "understanding. One thing is certain: the question is no longer whether AIs "
           "think, but how we will coexist with them.")

# Typed questions (choice / score / noul)
payload = {
    "tone": {
        "type": "choice",
        "instructions": "What is the dominant tone of `body`?",
        "criteria": {
            "neutral": "factual account, no judgment",
            "alarmist": "emphasis on risks and dangers",
            "enthusiastic": "emphasis on promises and benefits"
        }
    },
    "domain_coverage": {
        "type": "score",
        "instructions": "How many distinct application domains are covered in `body`?",
        "criteria": ["none", "one", "two", "three or more"]
    },
    "mentions_doubt": {
        "type": "noul",
        "instructions": "Does the text mention AIs' ability to doubt or flag their own uncertainty?"
    }
}

# One single forward pass for all questions
timer = time.perf_counter()
result = agent.predict({"body": article}, payload)

for name in payload:
    qtype = payload[name]["type"]
    print(payload[name]["instructions"])
    print("->", result["answers"][name][qtype], "\n")

print(f"Duration: {((time.perf_counter() - timer) * 1000):.2f} ms")
```

Expected output:

```
What is the dominant tone of `body`?
-> neutral

How many distinct application domains are covered in `body`?
-> 3

Does the text mention AIs' ability to doubt or flag their own uncertainty?
-> 0.98...
```

## Fine-tuning (French)

This snapshot is prepared for fine-tuning on French data:

- `rl_common.py` keeps the upstream RLCD training code (model construction, strictly-proper scoring rewards, metrics) as a reference.
- Default config: `max_len=512`, `head_max_len=192` — raise `agent.cfg["max_len"]` if your French documents are long (ModernBERT handles up to 8,192 tokens with RoPE).
- Tip: refit the calibration temperatures in `rl_agent_config.json` on your own French eval set before trusting the probabilities (see the upstream fine-tuning notebook).

## License

**Apache-2.0** — unchanged from the upstream [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) checkpoint. All credit for the model, training recipe (RLCD), and original documentation goes to Convai Innovations and the upstream authors.