Fix bug with empty slice if amount of dimensions matches amount of frequencies, but is not equal to 512
Currently, in FourierEmb4D if self.dimension != 512, but self.freqs ** 4 == self.dimensions // 2 (no harmonics should be cut off), model will have diff == 0 and a slice loc = loc[:, :, :-diff] will become loc = loc[:, :, :0] where ":0" part will return empty slice instead of all harmonics. Changed:
- Fixed empty slice bug: now slicing is only performed if diff > 0
- Added ValueError if diff < 0 during forward and similar ValueError if self.dimension > harmonics during init. Both cases represent the case where there are not enough harmonics for positional encoding)
- Fixed hardcoding special preprocessing only if self.harmonics != 512 (harmonics cut off) during forward. Now for any self.dimension the same cut off logic is generalized.
- Added ValueError during init if rqeuested self.dimension is not even as dimensionality of output Fourier embeddings is always even 2 * self.freqs ** 4 because of doubling while using both sine and cosine
- Added small comment of how and why is harmonics cut off performed
Hello, thank you for your contribution.
This problem has already been addressed in the braindecode version of the model: see modeling code.
We propose to unify the two by including your checks in the constructor and applying the modeling code from braindecode.
I reproduced the error and checked the fix for: (dimension, freqs) in (2, 1), (32, 2), (1250, 5), (2592, 6).
Let me know if this approach works for you!
Note that I will be closing the issue in brain-bzh/reve-large to avoid duplicates but the Large model itself will receive this unified fix.