| | --- |
| | license: apache-2.0 |
| | datasets: |
| | - Fishfishfishfishfish/Synthetic_text.txt |
| | language: |
| | - en |
| | --- |
| | the only files needed for inference is inference.py, word2idk.pkl, and lstm_Hxxx.safetensors |
| | |
| | input tokens must be space separated, as they aren't tokenized like the training data is. |
| | >python inference.py --temp 0.5 --top_k 64 --model_file lstm_H256.safetensors --start_sequence "User : what is the capital of France ? Bot : " --max_length 32 |
| |
|
| | usually results in something like |
| |
|
| | >The capital of the world of the world of the world of the world of the |
| |
|
| | its not very accurate yet, its trained on only 1.2mb of text |
| |
|
| | Each safetensors file represents a different hidden dim value. |
| | Each trained for 1 epoch. |
| |
|
| | inference.py hidden dim value must be edited for each safetensors. |
| |
|
| | >sequence_length = 64 |
| | > |
| | >batch_size = 16 |
| | > |
| | >learning_rate = 0.0001 |
| | > |
| | >embedding_dim = 256 |
| | > |
| | >num_layers = 4 |
| | > |
