---
license: apache-2.0
base_model: convaiinnovations/laya
library_name: llamadart
tags:
  - laya
  - decision-model
  - tetris
  - safetensors
---

# Laya Tetris head

A decision head for [Laya](https://huggingface.co/convaiinnovations/laya), fine-tuned to pick Tetris placements. It is used by llamadart's [`laya_tetris` example](https://github.com/leehack/llamadart/tree/main/example/laya_tetris) through `DecisionEngine` ([llamadart#604](https://github.com/leehack/llamadart/issues/604)).

`laya-head-tetris.safetensors` holds the 36 head tensors under Laya's PyTorch names, in F32, with no encoder tensors. Its `laya.config` metadata is the unmodified `rl_agent_config.json` of `convaiinnovations/laya` at revision `1c5edc17a7acd8701df6fc341c0d179f1c62c982`, so `DecisionEngine.load` needs no `configPath`. Pair it with a Laya ModernBERT backbone GGUF, such as `laya-Q8_0.gguf` from [`fr0stbit3/laya-gguf`](https://huggingface.co/fr0stbit3/laya-gguf).

```dart
final decisions = await DecisionEngine.load(
  engine, // a LlamaEngine with the Laya backbone GGUF loaded
  headPath: 'laya-head-tetris.safetensors',
);
```

## Training

- **Base:** the head of `convaiinnovations/laya` at `1c5edc17`, with the encoder frozen. Only `head.*`, `type_emb.*` and `scorer.*` were trained; the act head and the temperatures are unchanged.
- **Data:** 16,000 training and 2,000 validation `choice` questions from `bin/make_dataset.dart` in the example. Each question asks "Which placement of the falling Tetris piece is best?" and lists 2 to 6 candidate placements with their lines cleared, new holes, stack height and bumpiness. Targets are `softmax(h / 0.3)` over the example heuristic's value for each placement.
- **Recipe:** 12 epochs; AdamW at lr 3e-4, weight decay 0.01; 100 warmup steps, then cosine decay; batch 32. The best validation epoch (12) was kept. This is the default run of the example's fine-tuning notebook, proposed in [llamadart#619](https://github.com/leehack/llamadart/pull/619).

## Results

On the 2,000 validation questions: accuracy counts a pick among the heuristic-best placements as correct; regret is the heuristic value lost against the best placement.

| Head | Accuracy | Regret |
| --- | --- | --- |
| Uniform random pick | 0.275 | |
| Laya base head | 0.305 | 1.621 |
| This head, PyTorch F32 encoder | 0.757 | 0.234 |
| This head, llamadart with `laya-Q8_0.gguf` on Metal | 0.760 | 0.236 |

Turn-based games in the example's `bin/bench.dart`: `laya-Q8_0.gguf` on Metal, Apple M4 Max, mean over 5 seeds, capped at 150 pieces.

| Player | Candidates | Pieces | Lines | Questions per piece |
| --- | --- | --- | --- | --- |
| This head, one choice question | 3 heuristic-best + 3 random | 150 | 48 | 1 |
| This head, knockout rounds of choice questions | all legal placements | 135 | 40 | 4.8 |
| Base head, yes/no checklist | 3 heuristic-best + 3 random | 120 | 35 | 11.9 |
| Base head, one choice question | 3 heuristic-best + 3 random | 33 | 1 | 1 |
| Heuristic alone | all legal placements | 150 | 56 | |

With one choice question, every game reached the 150-piece cap.

## Revisions

- `83794e0bdd5526420997279a5d3dc9810a445217`: the first head, trained for 8 epochs. Validation accuracy 0.750 (PyTorch) and 0.747 (llamadart); 122 pieces and 34 lines with one choice question, 134 and 40 with knockout rounds.

## Limitations

- Trained only on the question format above. Other questions, including general Laya decisions, have not been evaluated with this head.
- English only, like the base model.

## License

Apache-2.0, like the base model. This is a modified version of the `convaiinnovations/laya` head: the weights listed under Training were fine-tuned on the Tetris data.