APE / README.md
Nitin2004's picture
Create README.md
5149290 verified

In order to use the model with

  1. For Rotary Position Embeddings, in model_args, set use_rope = True, use_ape = False, use_alibi = False
  2. For Attention with Linear Biases, in model_args, set use_rope = False, use_ape = False, use_alibi = True
  3. For Adaptive Position Encoding, in model_args, set use_rope = True, use_ape = True, use_alibi = False Upon downloading the weights, please convert it into a .pt file to be used for inference