| | --- |
| | license: mit |
| | language: |
| | - en |
| | --- |
| | |
| |
|
| | This is a very basic pyTorch transformer model that sorts lists of numbers. It was trained with nanoGPT. |
| |
|
| | The context window is 256 tokens, so the input list can be up to 127 tokens long. Numbers can be 0 to 99, separated by comma tokens. |
| |
|
| | It was trained for about one day on a laptop with a single NVIDIA RTX 2070 eGPU, so don't expect anything amazing. |
| | In practice it sorts these lists correctly about 90% of the time, which is good enough to satisfy my curiosity. |
| |
|
| | To run, I recommend cloning nanoGPT (https://github.com/karpathy/nanoGPT) and installing its prerequisites. |
| | Create a new branch and copy these files into the nanoGPT folder, overwriting the included sample.py and train.py. |
| |
|
| | To run: |
| |
|
| | > python sample.py --out_dir=out-sort-lists --start="(5,4,3,2,1): [" --num_samples=1 --temperature=0.0001 --max_new_tokens=127 |
| |
|
| | To train: |
| |
|
| | > python train.py config/train_sort.py |
| | |
