Positional Encoding

[mrluin]

Hello,

Thanks for your great work, an efficient and neat transformer framework is essential for low-level vision I think.

According to your work, I tried discard the attention mask and positional encoding in SwinIR, the training and inference speed is largely improved, and the attention mask has slight effect on performance. However, the performance severely droped after removing RPE in original SwinIR.

Could you please give me some hints about how can we discard RPE (and attention mask) correctly? directly removing codes related to positional encoding or removing positional encoding should incorporate someother necessary elements?

Looking forward to your reply, thanks~

[bonlime]

It probably works for authors because they add Shift-Conv (which is almost a conv3x3), default SwinIR doesn't have any 3x3 convs so postitional embedding is required