Converted GlobalAttentionPoolingHead to use PyTorch

items, not values

Removed diagnostics

Removed unnecessary files

Use dictionary coprehensions to do padding and return dictionaries instead of default dictionaries

Implement max_lengths for CorpusRepeater

Forgot a len

TPUs need constant batch shapes

Completed script for testing consistency

Trainable => trainable

Ensure weights are trainable

Fixed name of argument

Tensor is logits

Removed extraneous self

The other layer returned a tuple as well

Low level RoBERTa layers don't necessarily return what I expect them to

Fixed typo

Needed more arguments

Arguments to Concatenate layer should be in a list

Fixed arguments to decoder head

Incoplete testing script for consistency. Fixed typo

Testing script for reasoning

Testing scripts

Attention masks, generation, and testing script

Attention masks are only necessary for inputs

Making sure RoBERTa layers have all required arguments

Add an extra dimension to the input vector

Final dot product

dot_prod needs to unpack arguments from tuple

Removed unnecessary complication

Only inner function needs decorator

More vectorized_map weirdness

tensorflow.vectorized_map might not like getting function arguments in a tuple

Broadcasting dot products

Broadcasting dot products

Broadcasting dot products

Removed unnecessary ()

Forgot ()

Another workaround

itertools.batched not available

Lazy batching to prevent memory errors

Fixed indentation error

Error in invocation of tensordot

Encoding.pad modifies in place

Fix typo

columns is a list

columns is a local variable

Don't forget the label

Fix typo

Fix typo