Ensure tokenizer is on GPU

Diagnostics

Diagnostics

Diagnostics

Diagnostics

Fixed import

Fixed import

Factorized the weight matrix in the GlobalAttentionPoolingHead, thus reducing the number of parameters in this layer by a factor of 48

Correct dimension of consistency cosine

Using torch.nn.CosineSimilarity to simplify code

Removed unnecessary parameters

Attention mask in decoder

Set use_cache argument

Fix Einstein summation notation

Use keepdim option when normalising vectors

Make EPSILON a tensor

torch.maximum, not torch.max

Unsqueeze attention mask

Unpack BatchEncoding

input_embeddings not needed

Removed unnecessary parameter

get_input_embeddings() directly from base model

Missing 'from_pretrained'

config didn't need to be a property

There's a simpler way of doing this, I hope

Might be simpler to inherit from RobertaModel rather than PreTrainedModel

Removed a base model that was causing a loop in model initialisation

Problems with config

Removed line that would have failed

Fixed import

Typo

Further changes for compatibility with HuggingFace Pytorch implementation

PyTorch implementation of HugggingFace PreTrainedModel class does not allow direct setting of base_model. Rejig constructors accordingly

Removed superfluous ()

Corrected inheritance

Converted QaracTrainerModel to use PyTorch

Converted QaracDecoderModel to use PyTorch

Converted QaracEncoderModel to use PyTorch

Converted GlobalAttentionPoolingHead to use PyTorch

Trainable => trainable

Ensure weights are trainable

Removed extraneous self

The other layer returned a tuple as well

Low level RoBERTa layers don't necessarily return what I expect them to

Fixed typo

Needed more arguments

Arguments to Concatenate layer should be in a list

Fixed arguments to decoder head

Attention masks, generation, and testing script

Making sure RoBERTa layers have all required arguments