brain-bzh

/

reve-large

like 7

Follow

Broader Artificial Intelligence 23

Currently, in FourierEmb4D if self.dimension != 512, but self.freqs ** 4 == self.dimensions // 2 (no harmonics should be cut off), model will have diff == 0 and a slice loc = loc[:, :, :-diff] will become loc = loc[:, :, :0] where ":0" part will return empty slice instead of all harmonics. Changed:

1. Fixed empty slice bug: now slicing is only performed if diff > 0
2. Added ValueError if diff < 0 during forward and similar ValueError if self.dimension > harmonics during init. Both cases represent the case where there are not enough harmonics for positional encoding)
3. Fixed hardcoding special preprocessing only if self.harmonics != 512 (harmonics cut off) during forward. Now for any self.dimension the same cut off logic is generalized.
4. Added ValueError during init if rqeuested self.dimension is not even as dimensionality of output Fourier embeddings is always even 2 * self.freqs ** 4 because of doubling while using both sine and cosine
5. Added small comment of how and why is harmonics cut off performed