Using torch.nn.CosineSimilarity to simplify code

Fix Einstein summation notation

Use keepdim option when normalising vectors

Make EPSILON a tensor

torch.maximum, not torch.max

Unsqueeze attention mask

Converted GlobalAttentionPoolingHead to use PyTorch

Trainable => trainable

Ensure weights are trainable

Making sure RoBERTa layers have all required arguments

Final dot product

dot_prod needs to unpack arguments from tuple

Removed unnecessary complication

Only inner function needs decorator

More vectorized_map weirdness

tensorflow.vectorized_map might not like getting function arguments in a tuple

Broadcasting dot products

Broadcasting dot products

Broadcasting dot products

Error in invocation of tensordot

More work on models

GlobalAttentionPoolingHead layer

Datasets documentation

Code for building and training base modkels.

Code for building and training base modkels.