Computation of HSIC

[Pirazh]

Hi,

Thank you for your great work. I was going through the code and notice that here you would like to make only the diagonal elements zero which is consistent with the original paper "Feature Selection via Dependence Maximization". However, torch.diag function which you used here, has different behaviors when the input is vector or a matrix. Since here it is taking a matrix as the input, its output is a vector containing diagonal elements and kernel_XX - torch.diag(kernel_XX) subtracts the diagonal elements from all the elements in the corresponding column. The fix would be to change this line to kernel_XX - torch.diag(torch.diag(kernel_XX)). Could you please confirm this?

[Cogito2012]

@Pirazh Thanks for pointing out this issue. You are right after I referred to the torch.diag definition from pytorch doc. Since my HSIC omputation was adapted from the ICML'21 work ReBias where their HSIC code is here, I believe they have exactly the same issue as this one. I'll update the code later.