Arcface Code vs Paper Understanding

[ElisonSherton]

Hey guys!

Good Day! Hope everyone's keeping well. I had one question regarding the computation of cosine of target logit in ArcFace loss.

In the source-code for arcface loss, I found the following

  def modify_cosine_of_target_classes(self, cosine_of_target_classes):
      angles = self.get_angles(cosine_of_target_classes)
      return torch.cos(angles + self.margin)

There is an addition of margin to the angle and then we compute the cosine. While this is alright for all cases where the feature embedding makes an angle less than 180 deg - margin deg, when our angle is higher, we would be in the third quadrant and the cos value becomes less negative i.e. increases there. So, we would want to penalize our model more for highly obtuse angles but when the angle becomes too large, our penalty starts to soften. Hence the authors of the insightface paper in their repo insightface handle it as follows

final_target_logit = torch.where(target_logit > self.theta, cos_theta_m, target_logit - self.sinmm)

This is to keep the loss function monotonically decreasing with respect to increasing angles between target center and the feature embedding.

However, despite this change, pml's arcface gives decent results; any ideas on what is going on here or did I miss any code where you have added this check already?

Thanks a lot,
Vinayak.

[ElisonSherton]

The repo author has marked this as a bug and will be considered. Hence closing this discussion. Thanks Kevin!