|
|
--- |
|
|
language: |
|
|
- en |
|
|
- fr |
|
|
library_name: transformers |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- binary-carry-probagation |
|
|
- binary-level |
|
|
- bit-level |
|
|
- causal-lm |
|
|
- tokenizer-free |
|
|
- base2 |
|
|
- binary |
|
|
- calculator |
|
|
- addition |
|
|
- TinyTransformerLM |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- PhysiQuanty/BINARY-CARRY-PROPAGATION |
|
|
--- |
|
|
|
|
|
# Binary-Calculator-LLM (Proof of Concept) |
|
|
|
|
|
A tiny **tokenizer-free / bit-level** (base-2) **calculator** proof of concept. |
|
|
|
|
|
This repository ships custom `modeling_*.py` / `configuration_*.py`, so you must load it with `trust_remote_code=True`. |
|
|
|
|
|
## What it does |
|
|
|
|
|
The model is trained to read two integers encoded as **10-bit binary** inside a structured prompt, and to emit an answer inside a `BOR ... EOR` block (binary output, variable-length). |
|
|
|
|
|
### Vocab (size = 8) |
|
|
|
|
|
- Bits: `0`, `1` |
|
|
- Specials: |
|
|
- `BOS=2`, `EOS=3` |
|
|
- `BOI=4`, `EOI=5` (integer input blocks) |
|
|
- `BOR=6`, `EOR=7` (integer result block) |
|
|
|
|
|
## Load (Python) |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM |
|
|
|
|
|
m = AutoModelForCausalLM.from_pretrained( |
|
|
"PhysiQuanty/Binary-Calculator-LLM-POC", |
|
|
trust_remote_code=True, |
|
|
) |
|
|
m.eval() |
|
|
```` |
|
|
|
|
|
## Inference (CLI) |
|
|
|
|
|
This repo is typically used with the companion inference script `inference_binary_calculator3.py` (manual token-by-token loop, no `.generate()`), supporting: |
|
|
|
|
|
* `--prompt_int "int,int"` → builds: `BOS t0 t1 BOI <10b int1> EOI BOI <10b int2> EOI` |
|
|
* `--print_int` → extracts the **first** `BOR ... EOR` block and prints the decoded integer |
|
|
|
|
|
### Command |
|
|
|
|
|
```bash |
|
|
python3 inference_binary_calculator3.py \ |
|
|
--repo "PhysiQuanty/Binary-Calculator-LLM-POC" \ |
|
|
--prompt_int "20,68" \ |
|
|
--seed -1 \ |
|
|
--stop_on_eos \ |
|
|
--max_new_tokens 64 \ |
|
|
--temperature 0.7 \ |
|
|
--top_k 50 \ |
|
|
--print_int |
|
|
``` |
|
|
|
|
|
### Example output |
|
|
|
|
|
```text |
|
|
[Seed] 1011554894 |
|
|
[Device] cuda |
|
|
[Model] loaded from PhysiQuanty/Binary-Calculator-LLM-POC | vocab_size=8 |
|
|
[Prompt Origin] prompt_int="20,68" (t0,t1=0,0) |
|
|
[Prompt IDs] len=27 first32=[2, 0, 0, 4, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 5, 4, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 5] |
|
|
|
|
|
[Generated RAW IDS] |
|
|
[6, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 7, 6, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 7, 3] |
|
|
|
|
|
[Generated RAW IDS (as digits)] |
|
|
600001011000760000000100073 |
|
|
|
|
|
[PrintInt] First BOR..EOR |
|
|
[PrintInt] pos=27 nbits=11 bits=00001011000 int=88 |
|
|
``` |
|
|
|
|
|
## Notes |
|
|
|
|
|
* Inputs are **10-bit integers** (0..1023). The output can exceed 10 bits (e.g. addition overflow), so the `BOR..EOR` block is decoded with **variable bit-length**. |
|
|
* The model is **tokenizer-free** in the sense that it operates directly on bits and a tiny set of structural tokens. |
|
|
* This is a **POC**: sampling settings (`temperature`, `top_k`) can affect stability. For deterministic behavior, you can lower temperature and/or increase constraints. |
