File size: 2,733 Bytes
204af68 0d0c294 204af68 1f5048c 204af68 a6c916c 204af68 a6c916c 204af68 1f5048c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 |
---
language:
- en
- fr
library_name: transformers
pipeline_tag: text-generation
tags:
- binary-carry-probagation
- binary-level
- bit-level
- causal-lm
- tokenizer-free
- base2
- binary
- calculator
- addition
- TinyTransformerLM
license: apache-2.0
datasets:
- PhysiQuanty/BINARY-CARRY-PROPAGATION
---
# Binary-Calculator-LLM (Proof of Concept)
A tiny **tokenizer-free / bit-level** (base-2) **calculator** proof of concept.
This repository ships custom `modeling_*.py` / `configuration_*.py`, so you must load it with `trust_remote_code=True`.
## What it does
The model is trained to read two integers encoded as **10-bit binary** inside a structured prompt, and to emit an answer inside a `BOR ... EOR` block (binary output, variable-length).
### Vocab (size = 8)
- Bits: `0`, `1`
- Specials:
- `BOS=2`, `EOS=3`
- `BOI=4`, `EOI=5` (integer input blocks)
- `BOR=6`, `EOR=7` (integer result block)
## Load (Python)
```python
from transformers import AutoModelForCausalLM
m = AutoModelForCausalLM.from_pretrained(
"PhysiQuanty/Binary-Calculator-LLM-POC",
trust_remote_code=True,
)
m.eval()
````
## Inference (CLI)
This repo is typically used with the companion inference script `inference_binary_calculator3.py` (manual token-by-token loop, no `.generate()`), supporting:
* `--prompt_int "int,int"` → builds: `BOS t0 t1 BOI <10b int1> EOI BOI <10b int2> EOI`
* `--print_int` → extracts the **first** `BOR ... EOR` block and prints the decoded integer
### Command
```bash
python3 inference_binary_calculator3.py \
--repo "PhysiQuanty/Binary-Calculator-LLM-POC" \
--prompt_int "20,68" \
--seed -1 \
--stop_on_eos \
--max_new_tokens 64 \
--temperature 0.7 \
--top_k 50 \
--print_int
```
### Example output
```text
[Seed] 1011554894
[Device] cuda
[Model] loaded from PhysiQuanty/Binary-Calculator-LLM-POC | vocab_size=8
[Prompt Origin] prompt_int="20,68" (t0,t1=0,0)
[Prompt IDs] len=27 first32=[2, 0, 0, 4, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 5, 4, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 5]
[Generated RAW IDS]
[6, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 7, 6, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 7, 3]
[Generated RAW IDS (as digits)]
600001011000760000000100073
[PrintInt] First BOR..EOR
[PrintInt] pos=27 nbits=11 bits=00001011000 int=88
```
## Notes
* Inputs are **10-bit integers** (0..1023). The output can exceed 10 bits (e.g. addition overflow), so the `BOR..EOR` block is decoded with **variable bit-length**.
* The model is **tokenizer-free** in the sense that it operates directly on bits and a tiny set of structural tokens.
* This is a **POC**: sampling settings (`temperature`, `top_k`) can affect stability. For deterministic behavior, you can lower temperature and/or increase constraints. |