README.md

---
license: mit
tags:
- pi
- mlp
---
# pi-predicter

A neural network model that memorizes and predicts digits of π (pi) using Fourier feature encoding. This model demonstrates the memorization capabilities of MLPs with positional encodings by learning to predict specific digits of π at given positions.

## Model Description

**pi-predicter** is a multi-layer perceptron (MLP) that takes a position index as input and predicts the corresponding digit of π at that position. The model uses Fourier feature encoding to transform integer positions into high-dimensional representations, enabling the network to memorize up to 100,000 digits of π.

### Key Features:
- **Fourier Feature Encoding**: 16 frequency components for positional encoding
- **Deep Architecture**: 3 hidden layers with 512 dimensions each
- **Large Capacity**: Trained on 100,000 digits of π
- **Efficient Inference**: Instant digit prediction at any position within training range

## Model Architecture

```
PiPredictor(
  ├── Input: position (integer)
  ├── Fourier Encoding: 16 frequencies → 32-dim vector
  ├── MLP: 32 → 512 → 512 → 512 → 10
  └── Output: digit probability distribution (0-9)
)
```

## Intended Use

### Primary Use Case
Memorization and retrieval of π digits for positions 1-100,000 (after decimal point). The model serves as a demonstration of neural network memorization capabilities and positional encoding techniques.

### Limitations
- **Position Range**: Only valid for positions 1-100,000 (training range)
- **No Generalization**: Cannot predict digits beyond training range
- **Memorization Only**: Not designed for mathematical computation or pattern discovery
- **Position 0**: Integer part (3) not included in training

## How to Use

### Installation
```bash
pip install torch
```

### Quick Start
```python
import torch
from model import PiPredictor

# Load model
checkpoint = torch.load('model.pt', map_location='cpu')
config = checkpoint['config']

# Initialize model
model = PiPredictor(
    max_pos=checkpoint['model_max_pos'],
    num_frequencies=config['num_frequencies'],
    hidden_dims=config['hidden_dims'],
    dropout=config['dropout'],
    encoding=config['encoding'],
    embedding_dim=config['embedding_dim']
)

model.load_state_dict(checkpoint['model_state'])
model.eval()

# Predict digit at position 2 (should be 4: π = 3.14159...)
position = torch.tensor([2], dtype=torch.long)
with torch.no_grad():
    logits = model(position)
    predicted_digit = torch.argmax(logits, dim=-1).item()
    
print(f"Digit at position 2: {predicted_digit}")  # Output: 4
```

### Batch Inference
```python
# Predict multiple positions at once
positions = torch.tensor([1, 2, 3, 4, 5, 10, 100], dtype=torch.long)
with torch.no_grad():
    logits = model(positions)
    predictions = torch.argmax(logits, dim=-1)
    
# π = 3.1415926535...
# positions 1-5: [1, 4, 1, 5, 9]
print(predictions.tolist())  # [1, 4, 1, 5, 9, 5, 9]
```

## Technical Notes

### Fourier Feature Encoding
The model uses Fourier features to transform scalar positions into high-dimensional vectors:
- 16 frequency components create a 32-dimensional embedding
- Frequencies are likely sampled from a Gaussian distribution
- This encoding enables the MLP to learn high-frequency functions of position

### Memory Efficiency
Despite memorizing 100,000 digits, the model achieves this with only ~550k parameters, demonstrating the efficiency of neural networks as lookup tables for structured data.

## License

This model is released under the MIT License.

---

**Note**: This model is intended for educational and demonstration purposes, showcasing neural network memorization capabilities rather than mathematical computation. The digits of π are deterministic and can be computed exactly using algorithms; this model demonstrates an alternative approach using machine learning techniques.