| In order to use the model with | |
| 1. For Rotary Position Embeddings, in model_args, set use_rope = True, use_ape = False, use_alibi = False | |
| 2. For Attention with Linear Biases, in model_args, set use_rope = False, use_ape = False, use_alibi = True | |
| 3. For Adaptive Position Encoding, in model_args, set use_rope = True, use_ape = True, use_alibi = False | |
| Upon downloading the weights, please convert it into a .pt file to be used for inference | |