tier-9 bit-serial reducer (L=1024), official scorer 0.902 local

README.md

@@ -0,0 +1,19 @@
+# bitserial-modmul-v7 (tier 9, L=1024)
+
+Submission for the SAIR Modular Arithmetic Challenge. One shared, p-conditioned recurrent cell
+in a fixed bit-serial Horner loop computes (a * b) mod p; the cell learns the per-step transition
+s' = (2*s + d*x) mod p and the loop only sequences bits. entry_class model.BitSerialReducer,
+output_base 2, ~471K params, L=1024 (warm-started from the L=512 cell).
+
+## Local evaluation (official open-source scorer, run locally; not the organizers' leaderboard)
+- modchallenge evaluate (full 1100, public/default seed, A100-class hardware): tiers 1-9 = 1.00,
+  highest tier 9, overall 0.902, wall-clock 139s (inside the 300s budget on this hardware; the
+  organizers' sandbox may be slower).
+- Randomising the weights collapses every solved tier to 0.00 (capability is in the trained parameters).
+
+## Limitation (honest)
+Not exact. The per-step map is exact on random states and exhaustively on primes < 64, but a sparse
+set of structured-trajectory inputs (powers-of-two-times-random) still fails at large primes,
+reproducing the Neural GPU limitation (Price, Zaremba, Sutskever 2016). The benchmark tiers reflect
+average-case accuracy on the official scorer's random-operand distribution, not worst-case exactness.
+Not organizer-verified; compliance ruling on the loop scaffold pending.

manifest.json

@@ -0,0 +1,7 @@
+{
+  "entry_class": "model.BitSerialReducer",
+  "output_base": 2,
+  "framework": "pytorch",
+  "model_description": "One shared, p-conditioned recurrent transition cell (~471K-param bidirectional 2-layer GRU + control-bit embedding + per-bit head) applied in a fixed bit-serial Horner loop at 1024-bit state width. Same weights reduce a mod p, reduce b mod p (multiplicand 1), and multiply the residues (multiplicand a mod p, control bits scanning b mod p); state carried as bits, modulus fed via 32-bit limbs. Output base-2; the scorer's decoder reconstructs the integer. Trained regime: primes below 2^1024, operands up to ~2048 bits (tiers 1-9); outside it the model abstains and emits a single zero.",
+  "training_description": "Warm-started from the L=512 dagger2 cell (the transition is width-agnostic) and fine-tuned at L=1024 on one-step transitions s' = (2*s + d*x) mod p (bit-length stratified, wrap-boundary oversampled; AdamW, lr warmup + cosine decay, best-by-validation). No precomputed tables, no hand-coded reduction or multiplication. Randomising the weights collapses every solved tier to 0. Evaluation primes are unseen during training."
+}

model.py

@@ -0,0 +1,161 @@
+"""Bit-serial learned reducer (general width) for the Modular Arithmetic Challenge.
+
+Same design as bit-serial-v1/v2: one shared, p-conditioned transition cell that
+learned s' = (2*s + d*x) mod p, applied in a fixed bit-serial Horner loop (reduce a,
+reduce b, multiply). The arithmetic is in the trained cell; the loop only sequences
+bits. Randomising the weights collapses accuracy to chance.
+
+This version generalises the state width to L (read from the checkpoint), so it
+covers tiers up to whatever L the weights were trained for. Bit extraction uses
+32-bit limbs (`to_bits_limbs`) so a modulus p >= 2^63 never overflows an int64
+tensor (needed at L >= 64). State is carried as bits between steps; the harness
+decoder reconstructs the integer answer from the emitted base-2 digits.
+
+Regime: primes p < 2^L and operands up to 4*L bits. Outside it the model abstains
+and emits [0] -- the honest fallback.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import torch
+from torch import nn
+
+from modchallenge.interface.base_model import ModularMultiplicationModel
+
+_MASK32 = (1 << 32) - 1
+
+
+def _to_bits_small(vals: torch.Tensor, width: int) -> torch.Tensor:
+    shifts = torch.arange(width - 1, -1, -1, device=vals.device)
+    return (vals[:, None] >> shifts[None, :]) & 1
+
+
+def to_bits_limbs(ints, dev, width: int) -> torch.Tensor:
+    """List of python ints (< 2^width) -> (N, width) MSB-first bit tensor via 32-bit limbs.
+
+    Overflow-safe for any width: no int64 tensor ever holds a value >= 2^32."""
+    nl = (width + 31) // 32
+    cols = []
+    for k in range(nl - 1, -1, -1):  # most-significant limb first
+        limb = torch.tensor([(v >> (32 * k)) & _MASK32 for v in ints],
+                            dtype=torch.int64, device=dev)
+        cols.append(_to_bits_small(limb, 32))
+    bits = torch.cat(cols, dim=1)
+    return bits[:, nl * 32 - width:] if width < nl * 32 else bits
+
+
+class Cell(nn.Module):
+    def __init__(self, dmodel: int = 96, hidden: int = 128):
+        super().__init__()
+        self.in_proj = nn.Linear(3, dmodel)
+        self.d_emb = nn.Embedding(2, dmodel)
+        self.gru = nn.GRU(dmodel, hidden, num_layers=2, batch_first=True, bidirectional=True)
+        self.head = nn.Linear(2 * hidden, 1)
+
+    def forward(self, feat, d):
+        x = self.in_proj(feat) + self.d_emb(d)[:, None, :]
+        h, _ = self.gru(x)
+        return self.head(h).squeeze(-1)
+
+
+def _bits_of(n: int) -> list[int]:
+    if n <= 0:
+        return [0]
+    out: list[int] = []
+    while n > 0:
+        out.append(n & 1)
+        n >>= 1
+    out.reverse()
+    return out
+
+
+class BitSerialReducer(ModularMultiplicationModel):
+    def __init__(self) -> None:
+        self.model: Cell | None = None
+        self.device: torch.device | None = None
+        self.L = 32
+
+    def load(self, model_dir: str) -> None:
+        if torch.cuda.is_available():
+            self.device = torch.device("cuda")
+        elif torch.backends.mps.is_available():
+            self.device = torch.device("mps")
+        else:
+            self.device = torch.device("cpu")
+        ckpt = torch.load(Path(model_dir) / "weights.pt", map_location=self.device, weights_only=True)
+        self.L = int(ckpt.get("L", 32))
+        self.model = Cell(**ckpt.get("config", {}))
+        self.model.load_state_dict(ckpt["state_dict"])
+        self.model.to(self.device)
+        self.model.eval()
+
+    def preprocess_a(self, a):
+        return _bits_of(int(a))
+
+    def preprocess_b(self, b):
+        return _bits_of(int(b))
+
+    def preprocess_p(self, p):
+        return int(p)
+
+    @torch.no_grad()
+    def predict_digits(self, a_enc, b_enc, p_enc):
+        return self.predict_digits_batch([(a_enc, b_enc, p_enc)])[0]
+
+    @torch.no_grad()
+    def predict_digits_batch(self, inputs):
+        L = self.L
+        max_op = 4 * L
+        out: list[list[int]] = [[0] for _ in inputs]
+        idx, a_lists, b_lists, p_vals = [], [], [], []
+        for i, (a_enc, b_enc, p_enc) in enumerate(inputs):
+            p = int(p_enc)
+            a_bits = list(a_enc)
+            b_bits = list(b_enc)
+            if p < 2 or p >= (1 << L) or len(a_bits) > max_op or len(b_bits) > max_op:
+                continue
+            idx.append(i)
+            a_lists.append(a_bits)
+            b_lists.append(b_bits)
+            p_vals.append(p)
+        if not idx:
+            return out
+        dev = self.device
+        p_bits = to_bits_limbs(p_vals, dev, L).float()
+        ra = self._reduce(a_lists, p_bits, dev)
+        rb = self._reduce(b_lists, p_bits, dev)
+        prod = self._mul(ra, rb, p_bits)
+        prod_list = prod.long().tolist()
+        for j, i in enumerate(idx):
+            out[i] = [int(x) for x in prod_list[j]]
+        return out
+
+    def max_batch_size(self) -> int:
+        return 256
+
+    def _step(self, s_bits, x_bits, p_bits, d):
+        feat = torch.stack([s_bits, x_bits, p_bits], dim=-1)
+        return (torch.sigmoid(self.model(feat, d)) > 0.5).float()
+
+    def _reduce(self, bit_lists, p_bits, dev):
+        n = len(bit_lists)
+        width = max(len(b) for b in bit_lists)
+        padded = torch.zeros((n, width), dtype=torch.long, device=dev)
+        for r, bl in enumerate(bit_lists):
+            if bl:
+                padded[r, width - len(bl):] = torch.tensor(bl, dtype=torch.long, device=dev)
+        s_bits = torch.zeros((n, self.L), device=dev)
+        x_bits = to_bits_limbs([1] * n, dev, self.L).float()
+        for pos in range(width):
+            s_bits = self._step(s_bits, x_bits, p_bits, padded[:, pos])
+        return s_bits
+
+    def _mul(self, ra_bits, rb_bits, p_bits):
+        n = ra_bits.shape[0]
+        s_bits = torch.zeros((n, self.L), device=ra_bits.device)
+        rb_long = rb_bits.long()
+        for k in range(self.L):
+            s_bits = self._step(s_bits, ra_bits, p_bits, rb_long[:, k])
+        return s_bits

weights.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:241f1d558f0d152d3a8a905d22098d242edc00332d7eb0521fd5f9361aac848b
+size 1887610