我的预测值仅在梯度下降时才会降低

问题描述

我目前正在编写梯度下降的实现，但遇到了一个问题，即我的预测值（y_hat）只会降低。即使在训练标签为1而不是0的情况下应该增加，它也永远不会增加。我的火车功能代码如下：

def sigma(self,a):
    ans = 1/(1+np.exp(-a))
    return ans

  def get_loss(self,y_i,y_hat):
    loss = -(y_i * np.log(y_hat) + (1 - y_i) * np.log(1 - y_hat))
    return loss

def train(self,X,y,step_size,num_iterations):
    b_0 = 0
    rows = X.shape[0]
    columns = X.shape[1]
    weights = np.zeros(columns)
    losses = []
    for iteration in range(num_iterations):
      #Step 1: calculate y hat for row 
      summation = 0
      summation_k = np.zeros(columns)
      total_loss = 0
      for i in range(rows):
        row_total = np.sum(np.multiply(X[i],weights))
        y_hat = self.sigma(b_0 + row_total)
        y_i = y[i]
        # print('y_i: ',y_i)
        # print('y_hat: ',y_hat)
        # print()
        total_loss += self.get_loss(y_i,y_hat)
        diff = y_i - y_hat
        summation += diff

        # summation_k_i = summation_k_i + X[i] * diff
        summation_k = np.add(summation_k,np.multiply(diff,X[i]))
        
      # Compute change for each weight based on errors,then update the weights
      # Update b_0
      b_0 = b_0 + step_size * ((1/rows) * (-summation))

      # Update b_k
      # for j in range(columns):
      #   weights[j] = weights[j] + step_size * ((1/rows) * (-summation_k[j]))
      weights = np.add(weights,np.multiply(summation_k,(-step_size/rows)))
      
      # Keeping track of average loss for each iteration.
      losses.append(total_loss/rows)
    
    self.weights = np.insert(weights,b_0)
    return np.array(losses)

运行此命令时，每行和每次迭代的y_hat值始终减小。我找不到导致此错误的错误。

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）