---
license: mit
library_name: pytorch
tags:
  - transformer
  - arithmetic
  - multiplication
  - algorithmic-reasoning
  - latent-algorithm-learning
---

# Lattice Multiplication Transformer

[Interactive Demo](https://huggingface.co/spaces/davidandroid/lattice-multiplication-transformer-demo) ·
[Source Code](https://github.com/he-man-david/learning-multiplication-with-lattice-representation-using-transformer)

This repository contains the final checkpoint for an approximately 11.7-million-parameter model studying latent algorithm learning through an algorithm-aligned input representation.

Multi-digit multiplication is used as a case study. A learned frontend transforms two operands into an N × N lattice of digit-pair representations, which is passed to a standard Transformer encoder-decoder. The model directly generates the final product without scratchpads, carry labels, local-product labels, diagonal sums, or other intermediate supervision.

## Results

The final checkpoint was trained for 600,000 optimization steps.

- 19,955 correct products across 20,000 square-length evaluation examples from 1 × 1 through 20 × 20
- 99.775% aggregate exact-match accuracy across those evaluations
- 100% exact-match accuracy from 1 × 1 through 15 × 15
- 96.5% exact-match accuracy and approximately 99.90% token accuracy at 20 × 20

## Included Files

```text
final_lattice_multiplication__11m.pt
lattice_multiplication_transformer.py
tokenizer.py
config.json
```

The checkpoint contains the model state, optimizer state, curriculum state, training history, and random-number-generator states. Inference requires only `checkpoint["model_state_dict"]`.

## Loading the Checkpoint

```python
import json
import sys
from pathlib import Path

import torch
from huggingface_hub import snapshot_download


model_directory = Path(
    snapshot_download(
        repo_id="davidandroid/lattice-multiplication-transformer"
    )
)

sys.path.insert(0, str(model_directory))

from lattice_multiplication_transformer import (
    LatticeMultiplicationLoopTransformer,
)


with open(
    model_directory / "config.json",
    encoding="utf-8",
) as config_file:
    model_config = json.load(config_file)

model = LatticeMultiplicationLoopTransformer(
    **model_config
)

checkpoint = torch.load(
    model_directory / "final_lattice_multiplication__11m.pt",
    map_location="cpu",
    weights_only=False,
)

model.load_state_dict(
    checkpoint["model_state_dict"]
)

model.eval()
```

## Limitations

The model was trained on operands containing at most 20 digits. It does not achieve reliable exact-match length generalization beyond that range, even though token accuracy remains higher immediately beyond the training boundary.

This is a research model for studying latent algorithm learning, not a replacement for deterministic multiplication software.