---
language:
- en
- fr
library_name: transformers
pipeline_tag: text-generation
tags:
- binary-carry-probagation
- binary-level
- bit-level
- causal-lm
- tokenizer-free
- base2
- binary
- calculator
- addition
- TinyTransformerLM
license: apache-2.0
datasets:
- PhysiQuanty/BINARY-CARRY-PROPAGATION
---

# Binary-Calculator-LLM (Proof of Concept)

A tiny **tokenizer-free / bit-level** (base-2) **calculator** proof of concept.

This repository ships custom `modeling_*.py` / `configuration_*.py`, so you must load it with `trust_remote_code=True`.

## What it does

The model is trained to read two integers encoded as **10-bit binary** inside a structured prompt, and to emit an answer inside a `BOR ... EOR` block (binary output, variable-length).

### Vocab (size = 8)

- Bits: `0`, `1`
- Specials:
  - `BOS=2`, `EOS=3`
  - `BOI=4`, `EOI=5` (integer input blocks)
  - `BOR=6`, `EOR=7` (integer result block)

## Load (Python)

```python
from transformers import AutoModelForCausalLM

m = AutoModelForCausalLM.from_pretrained(
    "PhysiQuanty/Binary-Calculator-LLM-POC",
    trust_remote_code=True,
)
m.eval()
````

## Inference (CLI)

This repo is typically used with the companion inference script `inference_binary_calculator3.py` (manual token-by-token loop, no `.generate()`), supporting:

* `--prompt_int "int,int"` → builds: `BOS t0 t1 BOI <10b int1> EOI BOI <10b int2> EOI`
* `--print_int` → extracts the **first** `BOR ... EOR` block and prints the decoded integer

### Command

```bash
python3 inference_binary_calculator3.py \
  --repo "PhysiQuanty/Binary-Calculator-LLM-POC" \
  --prompt_int "20,68" \
  --seed -1 \
  --stop_on_eos \
  --max_new_tokens 64 \
  --temperature 0.7 \
  --top_k 50 \
  --print_int
```

### Example output

```text
[Seed] 1011554894
[Device] cuda
[Model] loaded from PhysiQuanty/Binary-Calculator-LLM-POC | vocab_size=8
[Prompt Origin] prompt_int="20,68" (t0,t1=0,0)
[Prompt IDs] len=27 first32=[2, 0, 0, 4, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 5, 4, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 5]

[Generated RAW IDS]
[6, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 7, 6, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 7, 3]

[Generated RAW IDS (as digits)]
600001011000760000000100073

[PrintInt] First BOR..EOR
[PrintInt] pos=27 nbits=11 bits=00001011000 int=88
```

## Notes

* Inputs are **10-bit integers** (0..1023). The output can exceed 10 bits (e.g. addition overflow), so the `BOR..EOR` block is decoded with **variable bit-length**.
* The model is **tokenizer-free** in the sense that it operates directly on bits and a tiny set of structural tokens.
* This is a **POC**: sampling settings (`temperature`, `top_k`) can affect stability. For deterministic behavior, you can lower temperature and/or increase constraints.