Use Tensor Cores for training and inference

[maxpain]

Hello. I have RTX 4090, can I utilise tensor cores for training the ECAPA TDNN Speaker ID model?

I tried this, but it didn't work.

torch.backends.cuda.matmul.allow_tf32 = True
torch.backends.cudnn.allow_tf32 = True

[TParcollet]

Hi, shouldn't tensorcores be automatically used when conditions are respected with fp16 training? I believe so.

[maxpain]

Is it safe to use a half-precision floating-point format instead of fp32?

[TParcollet]

fp16 and bf16 yes. If your tensors are multiple of 8, then they should use tensorcores.