> It is also worth to note that, generally speaking, an Encoder-Decoders of 2N parameters has the same compute cost as a decoder-only model of N parameters which gives it a different FLOP to parameter count ratio.
Can someone explain this to me? I'm not sure how the compute costs are the same between the 2N and N nets.
Can someone explain this to me? I'm not sure how the compute costs are the same between the 2N and N nets.