laya-typed-decisions-p150

Derived from convaiinnovations/laya-typed-decisions. Weights revision e929ae5c. This represents the model implementation on Tenstorrent hardware. See the original model card for license, training, and evaluation details.

Laya typed-decisions is the authors' fine-tune of Laya (ModernBERT-large encoder with an option-marker decision head, 421M parameters) on the LocalLLaMA/typed-decisions train split, with a 1024-token context and a 256-token head budget. One forward pass answers typed noul, choice and score questions with calibrated probabilities; it never generates text. This package serves the Jev-compatible /v1/systemone API of the authors' laya-serve on one Blackhole p150 chip, with a live demo at /demo/.

Runs on p150 or p150x4 — see the serve profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Architecture	ModernBERT-large encoder (28 layers, 1024 hidden, GeGLU, dual-theta RoPE, one global attention layer per three) + a 2-layer decision head with an option-marker scorer and an act head
Hardware	p150, p150x4
License	apache-2.0
Status	Experimental community bring-up

Intended use

Direct use: Calibrated typed decisions for the typed-decisions workflows (agent trace observability, customer service, invoice processing, security incidents).

Out-of-scope use: Text generation; non-English input; workflows far from the fine-tuning data without evaluation.

Quickstart

uv tool install tenstorrent   # once — the Tenstorrent CLI, `tt`
tt model pull tt-hous/laya-typed-decisions-p150
tt serve tt-hous/laya-typed-decisions-p150

tt model pull (or tt-model pull --with-weights) downloads the Docker image and the convaiinnovations/laya-typed-decisions weights at e929ae5cf69bc34259cd2f95c9e91145b818b1f0 (into your HF cache; they are not in the image). tt serve (or tt-model serve) starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.

Without tt-cli — tt-model alone does the whole job:

tt-model pull  tt-hous/laya-typed-decisions-p150 --with-weights
tt-model serve tt-hous/laya-typed-decisions-p150

Serve profiles

One image serves every profile below; pick one with --profile.

profile	hardware	mesh
p150 (default)	p150	P150
p150x4	p150x4	P150x4

Using it

This package exposes the model's own HTTP server on port 20000, not an OpenAI-compatible chat API — its request and response shapes are the model's own. See the author's notes above for the payload it expects.

Same API as tt-hous/laya-p150: POST /v1/systemone with {"state": ..., "questions": {id: {"type": "choice"|"score"|"noul", "instructions": ..., "criteria": ...}}, "max_len", "head_max_len", "min_confidence"} returns {"model", "answers": {id: {...}}, "usage": {input_tokens, output_tokens, state_tokens, state_tokens_dropped, truncated, truncated_questions}}. POST /v1/systemone/batch takes {"states": [...], "questions": {...}} (1 to 64 states sharing one question set) and returns {"results": [...], "total_usage": {...}}. Use max_len up to 1024 and head_max_len up to 256. GET /v1/health reports backend, precision, mesh, buckets and the startup sanity check (compared against server/sanity_reference_typed_decisions.json, selected by LAYA_SANITY_REFERENCE). Every /v1 response carries X-Inference-Time-Ms, X-Laya-Device-Ms and X-Laya-Batch. Existing laya and Jev clients work by changing the base URL. The demo page is served at /demo/. shim/laya_tt_backend.py installs a TtBackend on a pip laya Agent that routes agent.model.forward to this server's /v1/forward (served because LAYA_RAW_FORWARD=1).

Expected performance

Measured from the packaged image served on one p150 (build 2, image tt-model/laya-typed-decisions-p150:a4c8f7eecc1b, code sha256 7b36a18faed4998a61c5c6e355eb2bd62cdd9ca9f8ec408bf489b6c30def831b; builds 3 and 4 carried the same code; this build changes only the demo page files (demo/, server/demo.py and their tests), not the model or the API path; result files under /home/hous/dev/laya/evals/results/package_laya-typed-decisions-p150_p150_b2_20261006T022244Z, release notes in doc/release/RUN_NOTES.md). Typed-decisions test set (400 cases, 2,000 decisions, max_len 1024 / head_max_len 256): accuracy 0.764, soft accuracy 0.469, Brier 0.062, ECE 0.214, score MAE 0.244 (published 0.766 / 0.471 / 0.062 / 0.213 / 0.242; CPU fp32 on this host 0.766 / 0.471 / 0.061 / 0.213 / 0.242). Latency per call (client median, warm, authors' bench_latency protocol), p150 versus the authors' Tesla T4 for the English checkpoint: 1 question 10.8 ms (T4 39.5), 5 questions 24.5 ms (84.5), 10 questions 42.9 ms (158.6), 50 questions 197.6 ms (771); device forward 9.2 / 22.7 / 40.3 / 191.9 ms. Batched /v1/systemone/batch: 203 to 249 questions per second on one p150 (T4 103 to 332); the p150x4 profile 309 to 936. Long states that reach the 1024-token bucket (750-token rows, host-served): 23.9 / 99.0 / 183.2 / 305.6 ms for 1 / 5 / 10 / 16 questions. AG News 0.953 and DAIR Emotion 0.595 accuracy at the 1024 / 256 budget (the authors publish these suites for the English checkpoint only: 0.950 / 0.595). Parity against this checkpoint's CPU fp32 reference on 488 decisions: 383 of 383 confident decisions agree, 485 of 488 argmax agree, median max abs delta p 0.004, max 0.040.

Limitations

Fine-tuned for the four typed-decisions workflows (agent trace observability, customer service, invoice processing, security incidents); outside them use the base checkpoint. max_len 1024, head_max_len 256; requests are padded to 128, 256, 512 or 1024 tokens with 1 to 64 rows at 128 to 512 tokens (exact buckets at 5, 10 and 50 rows) and 1 to 16 rows at 1024 tokens (a longer 1024-token batch is split into chunks of 16 rows; at most 16384 padded tokens per forward); longer sequences are refused with 422. Temperatures are clamped to [0.5, 5.0] as pip laya 0.3.27 does, so choice questions with more than 10 options use 0.5 instead of the shipped 0.1006. action.act_probability carries no usable signal (authors' issue #185); gate on answer_confidence. 13 of 2,000 typed-decisions reference decisions flip on the device, all with a reference margin under 0.10 (max abs delta p 0.031); 3 of 488 parity decisions flip, all with a reference margin under 0.015 (none of the 383 confident decisions flips). The English bundle's served DAIR Emotion run showed single-row deviations against the fp32 reference of up to 0.27 with two confident flips (review R3); the corresponding rate is measured for this checkpoint as stated in limitations. One device process serializes requests. p150x4 status: served from the packaged sibling image (build 2: healthy in 20 s, 68.5 ms for 50 questions, parity smoke 100 of 100 and 81 of 81 confident). Not reproduced: MASSIVE, XNLI, 45 of 51 languages, post-temperature ECE.

Risks and safety considerations

bf8w_hifi3_erf numerics move probabilities by up to 0.040 (parity corpus, median 0.0039) and 0.031 (typed-decisions) against the fp32 reference in the packaged run (build 2); the single-row Emotion deviation seen on the English bundle is measured for this checkpoint as stated in limitations; gate on confidence, not act_probability. Long option texts are truncated to 48 tokens silently, as in the reference. The shipped temperatures are over-confident per the authors; refit on your own data.

Licensing

Laya typed-decisions weights and the vendored rl_common.py, rl_agent_api.py and email_utils.py: Apache-2.0 (Convai Innovations); ModernBERT-large: Apache-2.0 (Answer.AI); TTNN port and serving code: Apache-2.0 (Tenstorrent AI ULC); demo feed cases: LocalLLaMA/typed-decisions, Apache-2.0.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/tt-hous/laya-typed-decisions-p150/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: support@tenstorrent.com.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	a local checkout — commit not published
code/ digest	42ab6916a82d7364 (sha256, first 16 hex digits)
built	2026-10-06T13:50:55+00:00 by tt-model 0.1.0