In order to use the model with 
1. For Rotary Position Embeddings, in model_args, set use_rope = True, use_ape = False, use_alibi = False
2. For Attention with Linear Biases, in model_args, set use_rope = False, use_ape = False, use_alibi = True
3. For Adaptive Position Encoding, in model_args, set use_rope = True, use_ape = True, use_alibi = False
Upon downloading the weights, please convert it into a .pt file to be used for inference