用于 Spearman 等级相关性的 Hessians

编程问答 2022-05-16

问题描述

我尝试将 Spearman 等级相关系数 (wiki) 实现为 xgboost 的自定义目标函数。我正在使用 google 的 fast-soft-sort (github) 包进行可微排名和 tensorflow 来自动计算梯度。您可以在下面找到代码：

from fast_soft_sort.tf_ops import soft_rank
import tensorflow as tf
import numpy as np

def pearson_corr(x,y):
    
    xy_t = tf.concat([x,y],axis=0)
    mean_t = tf.reduce_mean(xy_t,axis=1,keepdims=True)
    cov_t = ((xy_t-mean_t) @ tf.transpose(xy_t-mean_t))/(x.shape[1]-1)
    cov2_t = tf.linalg.diag(1/tf.sqrt(tf.linalg.diag_part(cov_t)))
    corr_matrix = cov2_t @ cov_t @ cov2_t
    corr = tf.reduce_mean(corr_matrix) * 2 - 1 # equivalent to taking element [0][1] assuming the 2x2 corr matrix is symmetric and the diagonals are 1
    
    return corr

def spearman_corr(x,y):
    
    ranks = soft_rank(x,regularization_strength=0.1)
    corr = pearson_corr(ranks,y)
    
    return corr

def get_value_grad_and_hess(x,y,f):
    
    x_var = tf.Variable(x,dtype=tf.float32)
    y_var = tf.Variable(y,dtype=tf.float32)
        
    val,grad,hess = None,None,None

    with tf.GradientTape() as t2:
    
        with tf.GradientTape() as t1:
            
            val = f(x_var,y_var)
        
        grad = t1.gradient(val,x_var)    

    hess = t2.jacobian(grad,x_var)

    return val,hess

# test with random input
x = np.random.rand(1,10) # predictions
y = np.random.rand(1,10) # labels

print('pearson:')
val,hess = get_value_grad_and_hess(x,pearson_corr)
print(' value:',val)
print(' gradient:',grad)
print(' hessian:',hess)

print('spearman:')
val,spearman_corr)
print(' value:',hess)

示例输出：

pearson:
 value: tf.Tensor(-0.3348779,shape=(),dtype=float32)
 gradient: tf.Tensor(
[[ 0.21893269  0.16921082  0.19409613 -0.00321923  0.07347419  0.29004234
  -0.07947832 -0.7088071   0.29586902 -0.4501205 ]],shape=(1,10),dtype=float32)
 hessian: tf.Tensor(
[[[[ 0.04441248 -0.03097764  0.02028688 -0.20294864 -0.22516166
    -0.09771542 -0.06334648  0.42131865 -0.02681065  0.16094248]]

  [[-0.03097765  0.40132353  0.04399774 -0.07797898 -0.05632872
     0.04975905 -0.07172927 -0.17790946  0.06856277 -0.14871901]]

  [[ 0.02028689  0.04399772  0.44207606 -0.06522453 -0.03210837
     0.0911998  -0.07974204 -0.30411014  0.10508882 -0.22146425]]

  [[-0.20294863 -0.077979   -0.06522458  0.27985442 -0.12591925
    -0.13325104 -0.02723934  0.31153008 -0.10839472  0.14957213]]

  [[-0.22516167 -0.05632871 -0.03210838 -0.12591931  0.23029271
    -0.10794277 -0.04108595  0.30121914 -0.07069567  0.12773061]]

  [[-0.09771542  0.04975905  0.0911998  -0.13325103 -0.10794276
     0.4497667  -0.09163402 -0.12746409  0.11477053 -0.14748882]]

  [[-0.06334649 -0.07172926 -0.07974204 -0.02723937 -0.04108596
    -0.09163402  0.35762674  0.07487351 -0.09705587  0.03933275]]

  [[ 0.4213187  -0.17790946 -0.3041101   0.31153005  0.3012191
    -0.12746407  0.07487351 -0.09769349 -0.2807703  -0.12099396]]

  [[-0.02681071  0.06856281  0.1050889  -0.10839473 -0.07069571
     0.11477058 -0.0970559  -0.28077024  0.5259669  -0.23066193]]

  [[ 0.1609425  -0.14871901 -0.22146428  0.1495721   0.12773061
    -0.14748883  0.03933276 -0.12099396 -0.23066193  0.39175004]]]],10,1,dtype=float32)

spearman:
 value: tf.Tensor(-0.3408205,dtype=float32)
 gradient: tf.Tensor(
[[ 0.13679196  0.13627169  0.15643153 -0.10963751 -0.02715444  0.2698098
   0.20591483 -0.8303905   0.26787752 -0.20591483]],dtype=float32)
 hessian: None

如您所见，上面的代码为 pearson 相关函数生成了梯度和 hessian，但对于 Spearman 相关函数，hessian 为 None。

有人知道为什么 Hessian 对于 Spearman 相关性是 None 吗？

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）

gradientpythontensorflowtensorflowtensorflow