Timothy Do commited on
Update the model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: convaiinnovations/laya
|
| 4 |
+
library_name: sima-lmm
|
| 5 |
+
tags:
|
| 6 |
+
- sima.ai
|
| 7 |
+
- modalix
|
| 8 |
+
- mla
|
| 9 |
+
- edge
|
| 10 |
+
- decision-model
|
| 11 |
+
- modernbert
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# Laya decision models, compiled for the SiMa.ai MLSoC Modalix
|
| 15 |
+
|
| 16 |
+
[Laya](https://github.com/NandhaKishorM/laya) by Convai Innovations is a "System-1" decision
|
| 17 |
+
model: an encoder (ModernBERT-large, or mmBERT-base for the multilingual one) with a decision
|
| 18 |
+
head that answers a typed question about a piece of text in one forward pass. It picks one of
|
| 19 |
+
several options, scores on a scale, or says yes or no.
|
| 20 |
+
|
| 21 |
+
These are those checkpoints compiled for the Modalix MLA. Every graph is a single MLA stage,
|
| 22 |
+
with no layers on the CPU, and a decision takes about 19 ms at 128 tokens (8 ms for the
|
| 23 |
+
multilingual model).
|
| 24 |
+
|
| 25 |
+
| folder | model | precision | latency per decision, by tokens | same decision as PyTorch | size |
|
| 26 |
+
|---|---|---|---|---|---|
|
| 27 |
+
| `general-int8` | Laya INT8 | A_BF16_W_INT8 | 128: 14 ms | 98 / 100 | 0.67 GB |
|
| 28 |
+
|
| 29 |
+
Latency is one decision on a Modalix DevKit, measured inside the runtime. "Same decision" is
|
| 30 |
+
against the fp32 PyTorch model on 100 decisions with the 128-token graph. Precision `BF16` is
|
| 31 |
+
weights and activations in bfloat16; `A_BF16_W_INT8` keeps bfloat16 activations and stores
|
| 32 |
+
the weights as int8.
|
| 33 |
+
|
| 34 |
+
## What is in a folder
|
| 35 |
+
|
| 36 |
+
| file | what it is |
|
| 37 |
+
|---|---|
|
| 38 |
+
| `laya_s<N>_stage1_mla.elf` | the compiled graph for sequences of up to N tokens |
|
| 39 |
+
| `token_embeddings.bf16` | the embedding table, looked up on the CPU |
|
| 40 |
+
| `act_tail.f32` | the last layer of the act head, run on the CPU |
|
| 41 |
+
| `tokenizer.json` | the checkpoint's tokenizer |
|
| 42 |
+
| `laya_config.json` | token ids, calibration temperatures, and which graphs there are |
|
| 43 |
+
|
| 44 |
+
`models.json` lists every folder with file sizes and SHA-256 sums.
|
| 45 |
+
|
| 46 |
+
## Using them
|
| 47 |
+
|
| 48 |
+
The files run on a Modalix board with the runtime and web app from
|
| 49 |
+
[sima-laya](https://github.com/dotimothy/sima-laya): its Models page downloads a model from
|
| 50 |
+
this repository onto the board and loads it on the MLA. By hand:
|
| 51 |
+
|
| 52 |
+
```bash
|
| 53 |
+
hf download TDoSiMa/sima-laya --include "general/*" --local-dir .
|
| 54 |
+
./laya run general --state "We were billed twice for March." \
|
| 55 |
+
--question '{"type": "noul", "instructions": "Is this a billing problem?"}'
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
They are not usable with PyTorch or ONNX Runtime; for that, use the original checkpoints at
|
| 59 |
+
[convaiinnovations/laya](https://huggingface.co/convaiinnovations/laya).
|
| 60 |
+
|
| 61 |
+
## License and attribution
|
| 62 |
+
|
| 63 |
+
Apache-2.0. The models are Convai Innovations' Laya checkpoints
|
| 64 |
+
(https://github.com/NandhaKishorM/laya, Apache-2.0), compiled without retraining. The
|
| 65 |
+
exception is `dino`, whose decision head was fine-tuned on top of the English checkpoint for
|
| 66 |
+
a browser game. Zero-shot quality is the checkpoints': the port reproduces PyTorch's answers,
|
| 67 |
+
including its wrong ones.
|