File size: 442 Bytes
5149290 |
1 2 3 4 5 6 |
In order to use the model with
1. For Rotary Position Embeddings, in model_args, set use_rope = True, use_ape = False, use_alibi = False
2. For Attention with Linear Biases, in model_args, set use_rope = False, use_ape = False, use_alibi = True
3. For Adaptive Position Encoding, in model_args, set use_rope = True, use_ape = True, use_alibi = False
Upon downloading the weights, please convert it into a .pt file to be used for inference
|