Mini-Transformer / configs /model /test_model.yaml
AlaBoussoffara's picture
organized code and set up chainlit for demos
2d52135
Raw
History Blame Contribute Delete
321 Bytes
name: test_model
best_checkpoint_path: ./outputs/checkpoints/test_model/best.pt
latest_checkpoint_path: ./outputs/checkpoints/test_model/latest.pt
tokenizer: bpe_8k
d_model: 512
num_heads: 4
num_layers: 4
d_ff: 1024
dropout_rate: 0.1
vocab_size: 8192
max_seq_len: 128
pad_id: 0
bos_id: 1
eos_id: 2
layer_norm_style: post