Update README.md
Browse files
README.md
CHANGED
|
@@ -2,13 +2,13 @@
|
|
| 2 |
|
| 3 |
# Turing
|
| 4 |
|
| 5 |
-
Turing is a character-level AI language model based on the GCLM (Global
|
| 6 |
|
| 7 |
## Architecture
|
| 8 |
|
| 9 |
The model (`GCLM`) processes sequences using a stack of blocks that alternate between:
|
| 10 |
-
- **LocalConv1D**: Captures
|
| 11 |
-
- **GlobalConv1D**: Uses FFT to capture global context across the entire sequence length.
|
| 12 |
|
| 13 |
## Usage
|
| 14 |
|
|
@@ -30,6 +30,6 @@ python sample.py
|
|
| 30 |
```
|
| 31 |
|
| 32 |
## Requirements
|
| 33 |
-
- Python 3
|
| 34 |
-
- PyTorch
|
| 35 |
-
- tqdm
|
|
|
|
| 2 |
|
| 3 |
# Turing
|
| 4 |
|
| 5 |
+
Turing is a character-level AI language model based on the GCLM (Global Convolutional Language Model) architecture. It is designed to learn from text using a hybrid approach consisting of local 1-dimensional convolutions for short-range dependencies and FFT-based global 1D convolutions for long-range context.
|
| 6 |
|
| 7 |
## Architecture
|
| 8 |
|
| 9 |
The model (`GCLM`) processes sequences using a stack of blocks that alternate between:
|
| 10 |
+
- **LocalConv1D**: Captures local context (small chunks of n tokens)
|
| 11 |
+
- **GlobalConv1D**: Uses the FFT (Fast Fourier Transform) to capture global context across the entire sequence length.
|
| 12 |
|
| 13 |
## Usage
|
| 14 |
|
|
|
|
| 30 |
```
|
| 31 |
|
| 32 |
## Requirements
|
| 33 |
+
- Python 3 (install at https://python.org)
|
| 34 |
+
- PyTorch (run `pip install torch`)
|
| 35 |
+
- tqdm (`pip install tqdm`)
|