|  | |
| # Turing | |
| Turing is a character-level AI language model based on the GCLM (Global Convolutional Language Model) architecture. It is designed to learn from text using a hybrid approach consisting of local 1-dimensional convolutions for short-range dependencies and FFT-based global 1D convolutions for long-range context. | |
| ## Architecture | |
| The model (`GCLM`) processes sequences using a stack of blocks that alternate between: | |
| - **LocalConv1D**: Captures local context (small chunks of n tokens) | |
| - **GlobalConv1D**: Uses the FFT (Fast Fourier Transform) to capture global context across the entire sequence length. | |
| ## Usage | |
| ### Training | |
| To train the model on your own text data: | |
| 1. Place `.txt` files in the `data/` directory. | |
| 2. Run the training script: | |
| ```bash | |
| python train.py | |
| ``` | |
| This will automatically detect available hardware (CUDA, MPS, or CPU) and start training, saving checkpoints to `Turing_<params>.pt`. | |
| ### Inference | |
| To generate text, run: | |
| ```bash | |
| python sample.py | |
| ``` | |
| ## Requirements | |
| - Python 3 (install at https://python.org) | |
| - PyTorch (run `pip install torch`) | |
| - tqdm (`pip install tqdm`) |