PMLM is the language model described in Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order, which is trained with probabilistic masking. This is the "PMLM-R" variant, adapted from the authors' original implementation.