File size: 1,145 Bytes
53264fa c719bda 53264fa c719bda 53264fa c719bda |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
# Turing
Turing is a character-level AI language model based on the GCLM (Global Convolutional Language Model) architecture. It is designed to learn from text using a hybrid approach consisting of local 1-dimensional convolutions for short-range dependencies and FFT-based global 1D convolutions for long-range context.
## Architecture
The model (`GCLM`) processes sequences using a stack of blocks that alternate between:
- **LocalConv1D**: Captures local context (small chunks of n tokens)
- **GlobalConv1D**: Uses the FFT (Fast Fourier Transform) to capture global context across the entire sequence length.
## Usage
### Training
To train the model on your own text data:
1. Place `.txt` files in the `data/` directory.
2. Run the training script:
```bash
python train.py
```
This will automatically detect available hardware (CUDA, MPS, or CPU) and start training, saving checkpoints to `Turing_<params>.pt`.
### Inference
To generate text, run:
```bash
python sample.py
```
## Requirements
- Python 3 (install at https://python.org)
- PyTorch (run `pip install torch`)
- tqdm (`pip install tqdm`) |