| --- |
| license: apache-2.0 |
| library_name: transformers |
| pipeline_tag: text-classification |
| tags: [laya, minimal, snapshot, english-checkpoint, fine-tuning, french] |
| --- |
| |
| # Laya — Minimal Snapshot (English Checkpoint) |
|
|
| This repository is a **minimalist snapshot** of the [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) model hub. It keeps **only the English base checkpoint** (`laya`, ModernBERT-large backbone, 421M params) together with the files needed to run and fine-tune it. |
|
|
| **Upstream:** |
| - Model hub: [convaiinnovations/laya](https://huggingface.co/convaiinnovations/laya) |
| - Code & docs: [github.com/NandhaKishorM/laya](https://github.com/NandhaKishorM/laya) |
| - Package: `pip install laya` |
|
|
| ## What was removed |
|
|
| - `multilingual/` and `typed-decisions/` checkpoints (~1.4 GB of weights) |
| - Benchmark images and eval results (`assets/`, `eval/`) |
| - Legacy research scripts (`rl_agent_api.py`, `email_utils.py`) |
|
|
| The English weights themselves are **byte-identical to the upstream snapshot** — this repo does not modify, retrain, or re-quantize the original model in any way. It is simply a trimmed copy, ideal for pulling the model onto a device with limited disk space (~810 MB total, fully standalone and offline-capable). |
|
|
| The multilingual and typed-decisions checkpoints remain available from the upstream hub if you ever need them. |
|
|
| ## Layout |
|
|
| ``` |
| config.json model marker (architectures: LayaTypedDecisions) |
| model.safetensors full weights: encoder + decision heads (~804 MB) |
| rl_agent_config.json hyperparameters and calibration temperatures |
| encoder/config.json ModernBERT-large architecture config |
| tokenizer/ tokenizer.json + tokenizer_config.json |
| rl_common.py training reference (RLCD: model build, rewards, metrics) |
| test.py standalone smoke test |
| ``` |
|
|
| ## Quickstart |
|
|
| ```bash |
| pip install laya |
| ``` |
|
|
| The checkpoint is fully self-contained: the `model.safetensors` already embeds the ModernBERT-large encoder, so nothing is downloaded from the Hugging Face hub at load time. |
|
|
| ```python |
| import os |
| os.environ["HF_HUB_OFFLINE"] = "1" # run fully offline |
| |
| import laya |
| |
| agent = laya.load(".") # loads this directory |
| ``` |
|
|
| ## Concrete example (`test.py`) |
|
|
| Non-autoregressive typed decisions over a state, all questions answered in a **single forward pass**: |
|
|
| ```python |
| import os |
| os.environ["HF_HUB_OFFLINE"] = "1" |
| import laya |
| import time |
| |
| # Load the local checkpoint |
| agent = laya.load(".") |
| |
| # State to evaluate |
| article = ("By 2031, artificial intelligences no longer merely answer: they anticipate, " |
| "negotiate and sometimes push back. In newsrooms, hospitals and courtrooms, " |
| "autonomous agents propose decisions, draft summaries and flag their own " |
| "uncertainty. Some observers speak of a 'second cognitive wave': systems capable " |
| "of doubting, asking for clarification and refusing a poorly specified task. " |
| "Others worry about a silent dependency, where humans approve without truly " |
| "understanding. One thing is certain: the question is no longer whether AIs " |
| "think, but how we will coexist with them.") |
| |
| # Typed questions (choice / score / noul) |
| payload = { |
| "tone": { |
| "type": "choice", |
| "instructions": "What is the dominant tone of `body`?", |
| "criteria": { |
| "neutral": "factual account, no judgment", |
| "alarmist": "emphasis on risks and dangers", |
| "enthusiastic": "emphasis on promises and benefits" |
| } |
| }, |
| "domain_coverage": { |
| "type": "score", |
| "instructions": "How many distinct application domains are covered in `body`?", |
| "criteria": ["none", "one", "two", "three or more"] |
| }, |
| "mentions_doubt": { |
| "type": "noul", |
| "instructions": "Does the text mention AIs' ability to doubt or flag their own uncertainty?" |
| } |
| } |
| |
| # One single forward pass for all questions |
| timer = time.perf_counter() |
| result = agent.predict({"body": article}, payload) |
| |
| for name in payload: |
| qtype = payload[name]["type"] |
| print(payload[name]["instructions"]) |
| print("->", result["answers"][name][qtype], "\n") |
| |
| print(f"Duration: {((time.perf_counter() - timer) * 1000):.2f} ms") |
| ``` |
|
|
| Expected output: |
|
|
| ``` |
| What is the dominant tone of `body`? |
| -> neutral |
| |
| How many distinct application domains are covered in `body`? |
| -> 3 |
| |
| Does the text mention AIs' ability to doubt or flag their own uncertainty? |
| -> 0.98... |
| ``` |
|
|
| ## Fine-tuning (French) |
|
|
| This snapshot is prepared for fine-tuning on French data: |
|
|
| - `rl_common.py` keeps the upstream RLCD training code (model construction, strictly-proper scoring rewards, metrics) as a reference. |
| - Default config: `max_len=512`, `head_max_len=192` — raise `agent.cfg["max_len"]` if your French documents are long (ModernBERT handles up to 8,192 tokens with RoPE). |
| - Tip: refit the calibration temperatures in `rl_agent_config.json` on your own French eval set before trusting the probabilities (see the upstream fine-tuning notebook). |
|
|
| ## License |
|
|
| **Apache-2.0** — unchanged from the upstream [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) checkpoint. All credit for the model, training recipe (RLCD), and original documentation goes to Convai Innovations and the upstream authors. |
|
|
