Timothy Do commited on
Commit
db39c3d
·
verified ·
1 Parent(s): 8e49724

Update the model card

Browse files
Files changed (1) hide show
  1. README.md +67 -0
README.md ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: convaiinnovations/laya
4
+ library_name: sima-lmm
5
+ tags:
6
+ - sima.ai
7
+ - modalix
8
+ - mla
9
+ - edge
10
+ - decision-model
11
+ - modernbert
12
+ ---
13
+
14
+ # Laya decision models, compiled for the SiMa.ai MLSoC Modalix
15
+
16
+ [Laya](https://github.com/NandhaKishorM/laya) by Convai Innovations is a "System-1" decision
17
+ model: an encoder (ModernBERT-large, or mmBERT-base for the multilingual one) with a decision
18
+ head that answers a typed question about a piece of text in one forward pass. It picks one of
19
+ several options, scores on a scale, or says yes or no.
20
+
21
+ These are those checkpoints compiled for the Modalix MLA. Every graph is a single MLA stage,
22
+ with no layers on the CPU, and a decision takes about 19 ms at 128 tokens (8 ms for the
23
+ multilingual model).
24
+
25
+ | folder | model | precision | latency per decision, by tokens | same decision as PyTorch | size |
26
+ |---|---|---|---|---|---|
27
+ | `general-int8` | Laya INT8 | A_BF16_W_INT8 | 128: 14 ms | 98 / 100 | 0.67 GB |
28
+
29
+ Latency is one decision on a Modalix DevKit, measured inside the runtime. "Same decision" is
30
+ against the fp32 PyTorch model on 100 decisions with the 128-token graph. Precision `BF16` is
31
+ weights and activations in bfloat16; `A_BF16_W_INT8` keeps bfloat16 activations and stores
32
+ the weights as int8.
33
+
34
+ ## What is in a folder
35
+
36
+ | file | what it is |
37
+ |---|---|
38
+ | `laya_s<N>_stage1_mla.elf` | the compiled graph for sequences of up to N tokens |
39
+ | `token_embeddings.bf16` | the embedding table, looked up on the CPU |
40
+ | `act_tail.f32` | the last layer of the act head, run on the CPU |
41
+ | `tokenizer.json` | the checkpoint's tokenizer |
42
+ | `laya_config.json` | token ids, calibration temperatures, and which graphs there are |
43
+
44
+ `models.json` lists every folder with file sizes and SHA-256 sums.
45
+
46
+ ## Using them
47
+
48
+ The files run on a Modalix board with the runtime and web app from
49
+ [sima-laya](https://github.com/dotimothy/sima-laya): its Models page downloads a model from
50
+ this repository onto the board and loads it on the MLA. By hand:
51
+
52
+ ```bash
53
+ hf download TDoSiMa/sima-laya --include "general/*" --local-dir .
54
+ ./laya run general --state "We were billed twice for March." \
55
+ --question '{"type": "noul", "instructions": "Is this a billing problem?"}'
56
+ ```
57
+
58
+ They are not usable with PyTorch or ONNX Runtime; for that, use the original checkpoints at
59
+ [convaiinnovations/laya](https://huggingface.co/convaiinnovations/laya).
60
+
61
+ ## License and attribution
62
+
63
+ Apache-2.0. The models are Convai Innovations' Laya checkpoints
64
+ (https://github.com/NandhaKishorM/laya, Apache-2.0), compiled without retraining. The
65
+ exception is `dino`, whose decision head was fine-tuned on top of the English checkpoint for
66
+ a browser game. Zero-shot quality is the checkpoints': the port reproduces PyTorch's answers,
67
+ including its wrong ones.