Rename OPTIMALITY_INDEX to MAGNITUDE_INDEX, add CIRCUITS_TODO, add architecture search

- Rename OPTIMALITY_INDEX.md to MAGNITUDE_INDEX.md
- Change wording from "optimal" to "minimum magnitude found"
- Add CIRCUITS_TODO.md with 46 circuits to build
- Add prune_architecture method for flat 2-layer architecture search

CIRCUITS_TODO.md

@@ -0,0 +1,63 @@
+# Circuits TODO
+
+Threshold logic circuits to build.
+
+## Voting / Threshold Functions
+1. 1outof4
+2. 2outof4
+3. 3outof4
+4. atmost1outof4
+5. atmost2outof4
+6. atmost3outof4
+7. exactly1outof4
+8. exactly2outof4
+9. exactly3outof4
+10. majority3
+11. majority5
+12. majority7
+13. minority3
+14. minority5
+15. minority7
+
+## Comparison
+16. lessthanorequal
+17. greaterthanorequal
+18. comparator4bit
+
+## Encoders / Decoders
+19. 2to4decoder
+20. 4to2encoder
+21. 8to3encoder
+22. gray2binary
+23. binary2gray
+24. 7segment
+
+## Arithmetic
+25. carrylookahead4bit
+26. multiplier3x3
+27. multiplier4x4
+28. incrementer4bit
+29. decrementer4bit
+30. subtractor4bit
+31. negator4bit
+
+## Bit Manipulation
+32. popcount4
+33. popcount8
+34. clz4
+35. clz8
+36. ffs4
+37. reverse4
+38. reverse8
+
+## Shift / Rotate
+39. shiftleft4
+40. shiftright4
+41. rotateleft4
+42. rotateright4
+43. barrelshift4
+
+## Multiplexers / Demultiplexers
+44. mux8
+45. demux4
+46. demux8

OPTIMALITY_INDEX.md → MAGNITUDE_INDEX.md

@@ -1,15 +1,15 @@
-# Optimality Index
+# Magnitude Index
 
 Results of exhaustive magnitude enumeration on threshold logic circuits.
 
 ## Summary
 
-All circuits listed below have been verified via exhaustive enumeration. "Optimal" means no valid configuration exists at lower magnitude.
+All circuits listed below have been tested via exhaustive enumeration. "Min Mag" is the minimum magnitude at which valid configurations were found.
 
 ## Single-Layer Gates (Linearly Separable)
 
-| Circuit | Inputs | Params | Optimal Mag | Solutions | Configs Tested |
-|---------|--------|--------|-------------|-----------|----------------|
+| Circuit | Inputs | Params | Min Mag | Solutions | Configs Tested |
+|---------|--------|--------|---------|-----------|----------------|
 | threshold-not | 1 | 2 | 1 | 1 | 5 |
 | threshold-nor | 2 | 3 | 2 | 1 | 7 |
 | threshold-implies | 2 | 3 | 2 | 1 | 25 |
@@ -25,44 +25,46 @@ All circuits listed below have been verified via exhaustive enumeration. "Optima
 
 ## Multi-Layer Gates (Not Linearly Separable)
 
-| Circuit | Inputs | Params | Original Mag | Optimal Mag | Solutions | Reduction |
-|---------|--------|--------|--------------|-------------|-----------|-----------|
+| Circuit | Inputs | Params | Original Mag | Min Mag | Solutions | Reduction |
+|---------|--------|--------|--------------|---------|-----------|-----------|
 | threshold-xor | 2 | 9 | 10 | 7 | 6 | 30% |
 | threshold-xnor | 2 | 9 | 9 | 7 | 2 | 22% |
 | threshold-mux | 3 | 11 | 10 | 7 | 4 | 30% |
+| threshold-xor3 | 3 | 16 | 14 | 10 | 18 | 29% |
 
-## Optimized Variants Created
+## Magnitude-Minimized Variants
 
-These repos contain the magnitude-optimal weights:
+These repos contain the minimum-magnitude weights found:
 
 - `threshold-xor-mag7` - 6 solutions at magnitude 7
 - `threshold-xnor-mag7` - 2 solutions at magnitude 7
 - `threshold-mux-mag7` - 4 solutions at magnitude 7
+- `threshold-xor3-mag10` - 18 solutions at magnitude 10 (flat architecture)
 
 ## Pending / In Progress
 
 | Circuit | Params | Status |
 |---------|--------|--------|
-| threshold-halfadder | 12 | Running (expected optimal: 11) |
+| threshold-halfadder | 12 | Running (expected min: 11) |
 | threshold-mod4 | 9 | Running |
 | threshold-biimplies | 9 | Not yet tested (same as XNOR) |
 | threshold-halfsubtractor | 12 | Not yet tested |
 
 ## Methodology
 
-Exhaustive search enumerates all integer weight configurations by magnitude level (0, 1, 2, ...) until valid solutions are found. This guarantees the minimum magnitude is found.
+Exhaustive search enumerates all integer weight configurations by magnitude level (0, 1, 2, ...) until valid solutions are found. This finds the minimum magnitude within the search space.
 
 For circuits with >12 parameters, exhaustive search becomes impractical. Use evolutionary or simulated annealing instead.
 
 ## Key Findings
 
-1. **Single-layer threshold gates have unique optimal representations.** All tested single-layer gates have exactly 1 solution at their optimal magnitude.
+1. **Single-layer threshold gates appear to have unique minimum-magnitude representations.** All tested single-layer gates have exactly 1 solution at their minimum magnitude.
 
-2. **Multi-layer gates can have solution families.** XOR has 6 solutions at magnitude 7, organized into two structural families.
+2. **Multi-layer gates can have solution families.** XOR has 6 solutions at magnitude 7, XOR3 has 18 solutions at magnitude 10.
 
 3. **Non-linearly-separable functions benefit most from optimization.** XOR/XNOR/MUX achieved 22-30% magnitude reduction.
 
-4. **Optimal magnitude = sum of components for independent subnetworks.** Halfadder (XOR + AND) expected optimal is 7 + 4 = 11.
+4. **Architecture matters.** XOR3 flat architecture (mag 10) beats cascade architecture (mag 14) by 29%.
 
 ## Last Updated
 

prune.py

@@ -163,6 +163,11 @@ class Config:
     topology_remove_prob: float = 0.2
     topology_add_prob: float = 0.1
 
+    run_architecture: bool = False
+    arch_hidden_neurons: int = 3
+    arch_max_weight: int = 3
+    arch_max_mag: int = 20
+
     sensitivity_samples: int = 1000
 
     depth_max_collapse: int = 3
@@ -2921,6 +2926,231 @@ def prune_exhaustive_sparse(circuit: AdaptiveCircuit, evaluator: BatchedEvaluato
     )
 
 
+def prune_architecture(circuit: AdaptiveCircuit, evaluator: BatchedEvaluator, cfg: Config) -> PruneResult:
+    """
+    Architecture search - find optimal flat 2-layer architecture.
+
+    Searches for a flat architecture with N hidden neurons that computes
+    the same function as the circuit, potentially at lower magnitude.
+
+    Parameters controlled by:
+        cfg.arch_hidden_neurons: number of hidden neurons (default 3)
+        cfg.arch_max_weight: max absolute weight value (default 3)
+        cfg.arch_max_mag: max magnitude to search (default 20)
+    """
+    start = time.perf_counter()
+    original = circuit.stats()
+
+    n_hidden = cfg.arch_hidden_neurons
+    n_inputs = circuit.spec.inputs
+    n_outputs = circuit.spec.outputs
+    max_weight = cfg.arch_max_weight
+    max_mag = cfg.arch_max_mag
+
+    # Parameters: n_hidden * (n_inputs + 1) + n_outputs * (n_hidden + 1)
+    n_params = n_hidden * (n_inputs + 1) + n_outputs * (n_hidden + 1)
+
+    if cfg.verbose:
+        print(f"    [ARCH] Architecture search")
+        print(f"    [ARCH] Hidden neurons: {n_hidden}")
+        print(f"    [ARCH] Inputs: {n_inputs}, Outputs: {n_outputs}")
+        print(f"    [ARCH] Parameters: {n_params}")
+        print(f"    [ARCH] Max weight: {max_weight}, Max magnitude: {max_mag}")
+        print(f"    [ARCH] Searching by magnitude level...")
+
+    test_inputs = circuit.test_inputs
+    test_expected = circuit.test_expected
+
+    def eval_flat_architecture(configs: torch.Tensor) -> torch.Tensor:
+        """Evaluate batch of flat architecture configs."""
+        batch_size = configs.shape[0]
+
+        # Extract weights for hidden layer
+        idx = 0
+        hidden_weights = []
+        hidden_biases = []
+        for h in range(n_hidden):
+            w = configs[:, idx:idx+n_inputs]
+            idx += n_inputs
+            b = configs[:, idx:idx+1]
+            idx += 1
+            hidden_weights.append(w)
+            hidden_biases.append(b)
+
+        # Extract weights for output layer
+        output_weights = []
+        output_biases = []
+        for o in range(n_outputs):
+            w = configs[:, idx:idx+n_hidden]
+            idx += n_hidden
+            b = configs[:, idx:idx+1]
+            idx += 1
+            output_weights.append(w)
+            output_biases.append(b)
+
+        # Compute hidden activations for all test inputs
+        # test_inputs: [n_cases, n_inputs]
+        # hidden_weights[h]: [batch, n_inputs]
+        n_cases = test_inputs.shape[0]
+
+        hidden_acts = []
+        for h in range(n_hidden):
+            # [batch, 1, n_inputs] * [1, n_cases, n_inputs] -> sum -> [batch, n_cases]
+            act = (hidden_weights[h].unsqueeze(1) * test_inputs.unsqueeze(0)).sum(dim=2) + hidden_biases[h]
+            act = (act >= 0).float()
+            hidden_acts.append(act)
+
+        hidden_stack = torch.stack(hidden_acts, dim=2)  # [batch, n_cases, n_hidden]
+
+        # Compute output
+        outputs = []
+        for o in range(n_outputs):
+            out = (hidden_stack * output_weights[o].unsqueeze(1)).sum(dim=2) + output_biases[o]
+            out = (out >= 0).float()
+            outputs.append(out)
+
+        if n_outputs == 1:
+            predicted = outputs[0]
+            expected = test_expected.squeeze()
+        else:
+            predicted = torch.stack(outputs, dim=2)
+            expected = test_expected
+
+        correct = (predicted == expected.unsqueeze(0)).float().mean(dim=1)
+        if n_outputs > 1:
+            correct = correct.mean(dim=1)
+
+        return correct
+
+    # Partition-based enumeration
+    @lru_cache(maxsize=None)
+    def partitions(total: int, n_slots: int, max_val: int) -> list:
+        if n_slots == 0:
+            return [()] if total == 0 else []
+        if n_slots == 1:
+            return [(total,)] if total <= max_val else []
+        result = []
+        for v in range(min(total, max_val) + 1):
+            for rest in partitions(total - v, n_slots - 1, max_val):
+                result.append((v,) + rest)
+        return result
+
+    def signs_for_partition(partition: tuple) -> torch.Tensor:
+        n = len(partition)
+        nonzero_idx = [i for i, v in enumerate(partition) if v != 0]
+        k = len(nonzero_idx)
+
+        if k == 0:
+            return torch.zeros(1, n, device=cfg.device, dtype=torch.float32)
+
+        n_patterns = 2 ** k
+        configs = torch.zeros(n_patterns, n, device=cfg.device, dtype=torch.float32)
+
+        for i, idx in enumerate(nonzero_idx):
+            signs = ((torch.arange(n_patterns, device=cfg.device) >> i) & 1) * 2 - 1
+            configs[:, idx] = signs.float() * partition[idx]
+
+        return configs
+
+    def generate_at_magnitude(target_mag: int):
+        all_configs = []
+        for partition in partitions(target_mag, n_params, max_weight):
+            signed = signs_for_partition(partition)
+            all_configs.append(signed)
+        if all_configs:
+            return torch.cat(all_configs, dim=0)
+        return torch.zeros(0, n_params, device=cfg.device)
+
+    total_tested = 0
+    all_solutions = []
+    optimal_mag = None
+
+    for target_mag in range(1, max_mag + 1):
+        mag_start = time.perf_counter()
+
+        configs = generate_at_magnitude(target_mag)
+        n_configs = configs.shape[0]
+
+        if n_configs == 0:
+            continue
+
+        if cfg.verbose:
+            print(f"    Magnitude {target_mag}: {n_configs:,} configs...", end=" ", flush=True)
+
+        # Batch evaluate
+        batch_size = 500000
+        valid_configs = []
+
+        for i in range(0, n_configs, batch_size):
+            batch = configs[i:i+batch_size]
+            fitness = eval_flat_architecture(batch)
+            valid_mask = fitness >= cfg.fitness_threshold
+            if valid_mask.any():
+                valid_configs.extend(batch[valid_mask].cpu().tolist())
+
+        total_tested += n_configs
+        mag_time = time.perf_counter() - mag_start
+
+        if valid_configs:
+            if cfg.verbose:
+                print(f"FOUND {len(valid_configs)} solutions! ({mag_time:.1f}s)")
+
+            optimal_mag = target_mag
+            all_solutions = valid_configs
+
+            if cfg.verbose:
+                print(f"    [ARCH] Optimal magnitude: {optimal_mag}")
+                print(f"    [ARCH] Solutions found: {len(all_solutions)}")
+                print(f"    [ARCH] First solution:")
+                sol = all_solutions[0]
+                idx = 0
+                for h in range(n_hidden):
+                    w = sol[idx:idx+n_inputs]
+                    idx += n_inputs
+                    b = sol[idx]
+                    idx += 1
+                    print(f"      h{h+1}: w={[int(x) for x in w]}, b={int(b)}")
+                for o in range(n_outputs):
+                    w = sol[idx:idx+n_hidden]
+                    idx += n_hidden
+                    b = sol[idx]
+                    idx += 1
+                    print(f"      out{o+1}: w={[int(x) for x in w]}, b={int(b)}")
+
+            break
+        else:
+            if cfg.verbose:
+                print(f"none ({mag_time:.1f}s)")
+
+    elapsed = time.perf_counter() - start
+
+    if cfg.verbose:
+        print(f"    [ARCH COMPLETE]")
+        print(f"      - Configurations tested: {total_tested:,}")
+        print(f"      - Optimal magnitude: {optimal_mag if optimal_mag else 'none found'}")
+        print(f"      - Original magnitude: {original['magnitude']:.0f}")
+        if optimal_mag:
+            print(f"      - Reduction: {(1 - optimal_mag/original['magnitude'])*100:.1f}%")
+        print(f"      - Solutions: {len(all_solutions)}")
+        print(f"      - Time: {elapsed:.1f}s")
+
+    return PruneResult(
+        method='architecture',
+        original_stats=original,
+        final_stats=original,  # We don't change the original weights
+        final_weights=circuit.clone_weights(),
+        fitness=evaluator.evaluate_single(circuit.weights),
+        time_seconds=elapsed,
+        metadata={
+            'hidden_neurons': n_hidden,
+            'optimal_magnitude': optimal_mag,
+            'total_tested': total_tested,
+            'solutions_count': len(all_solutions),
+            'all_solutions': all_solutions[:100]
+        }
+    )
+
+
 def run_all_methods(circuit: AdaptiveCircuit, cfg: Config) -> Dict[str, PruneResult]:
     """Run all enabled pruning methods."""
 
@@ -2965,6 +3195,7 @@ def run_all_methods(circuit: AdaptiveCircuit, cfg: Config) -> Dict[str, PruneRes
         ('fanin', cfg.run_fanin, lambda: prune_fanin(circuit, evaluator, cfg)),
         ('exhaustive_mag', cfg.run_exhaustive_mag, lambda: prune_exhaustive_mag(circuit, evaluator, cfg)),
         ('exhaustive_sparse', cfg.run_exhaustive_sparse, lambda: prune_exhaustive_sparse(circuit, evaluator, cfg)),
+        ('architecture', cfg.run_architecture, lambda: prune_architecture(circuit, evaluator, cfg)),
         ('evolutionary', cfg.run_evolutionary, lambda: prune_evolutionary(circuit, evaluator, cfg)),
         ('annealing', cfg.run_annealing, lambda: prune_annealing(circuit, evaluator, cfg)),
     ]
@@ -3140,6 +3371,9 @@ def main():
     parser.add_argument('--fanin-target', type=int, default=4)
     parser.add_argument('--sparse-max-weight', type=int, default=3, help='Max weight magnitude for sparse search')
     parser.add_argument('--exhaustive-max-params', type=int, default=12, help='Max params for exhaustive search')
+    parser.add_argument('--arch-hidden', type=int, default=3, help='Number of hidden neurons for architecture search')
+    parser.add_argument('--arch-max-weight', type=int, default=3, help='Max weight for architecture search')
+    parser.add_argument('--arch-max-mag', type=int, default=20, help='Max magnitude to search for architecture')
 
     args = parser.parse_args()
 
@@ -3163,14 +3397,17 @@ def main():
         annealing_parallel_chains=args.sa_chains,
         fanin_target=args.fanin_target,
         sparse_max_weight=args.sparse_max_weight,
-        exhaustive_max_params=args.exhaustive_max_params
+        exhaustive_max_params=args.exhaustive_max_params,
+        arch_hidden_neurons=args.arch_hidden,
+        arch_max_weight=args.arch_max_weight,
+        arch_max_mag=args.arch_max_mag
     )
 
     if args.methods:
         all_methods = ['magnitude', 'zero', 'quantize', 'evolutionary', 'annealing',
                        'structural', 'topology', 'sensitivity', 'weight_sharing',
                        'depth', 'gate_subst', 'symmetry', 'fanin',
-                       'exhaustive_mag', 'exhaustive_sparse']
+                       'exhaustive_mag', 'exhaustive_sparse', 'architecture']
         for m in all_methods:
             setattr(cfg, f'run_{m}', False)
 
@@ -3191,7 +3428,8 @@ def main():
                 'sym': 'symmetry', 'symmetry': 'symmetry',
                 'fanin': 'fanin', 'fan': 'fanin',
                 'exhaustive_mag': 'exhaustive_mag', 'exh_mag': 'exhaustive_mag', 'exh': 'exhaustive_mag', 'brute': 'exhaustive_mag',
-                'exhaustive_sparse': 'exhaustive_sparse', 'exh_sparse': 'exhaustive_sparse', 'sparse': 'exhaustive_sparse'
+                'exhaustive_sparse': 'exhaustive_sparse', 'exh_sparse': 'exhaustive_sparse', 'sparse': 'exhaustive_sparse',
+                'architecture': 'architecture', 'arch': 'architecture'
             }
             if m in method_map:
                 setattr(cfg, f'run_{method_map[m]}', True)