I wish this was written with more care. None of the symbols are defined.
Worst of all, they use "channel dimension" in a sequence model. What even is a channel in a sequence of tokens?
This happens as soon as you have a single person with CNN background on the team and it makes zero sense.
What if you actually have channels in your data? What then?
Worst of all, they use "channel dimension" in a sequence model. What even is a channel in a sequence of tokens? This happens as soon as you have a single person with CNN background on the team and it makes zero sense. What if you actually have channels in your data? What then?