如何优化 xgboost？

问题描述

我想对 xgboost 分类器执行超参数调整。当我使用特定的超参数值时，我看到了一些错误。请告知调整超参数的正确方法，例如 max_feature、标准、损失等

def xgb_grid_search(X,y,nfolds):
    #create a dictionary of all values we want to test
    param_grid = {'learning_rate': (0.0001,0.001,0.01,0.05,0.1,0.15)
                 }
    # xgb model
    xgb_model=xgb.XGBClassifier()
    #use gridsearch to test all values
    xgb_gscv = gridsearchcv(xgb_model,param_grid,cv=nfolds)
    #fit model to data
    xgb_gscv.fit(X,y)
    return xgb_gscv.best_params_

解决方法

不要将值括在方括号 ('learning_rate': (0.0001,0.001,0.01,0.05,0.1,0.15)) 中，而是使用方括号 ('learning_rate': [0.0001,0.15])。以下是来自 this kaggle notebook 的示例：

def xgb_grid_search(X,y,nfolds):
    #create a dictionary of all values we want to test
    param_grid = { 'max_depth': [4,5,6],'min_child_weight':[4,'learning_rate': [0.05,0.5],'n_estimators': [20,50,100] }
    # decision tree model
    xgb_model=XGBClassifier()
    #use gridsearch to test all values
    xgb_gscv = GridSearchCV(xgb_model,param_grid,cv=nfolds)
    #fit model to data
    xgb_gscv.fit(X,y)
    print(xgb_gscv.best_params_)
    print(xgb_gscv.best_estimator_)
    print(xgb_gscv.best_score_)

本教程提供了相当全面的指南：https://www.analyticsvidhya.com/blog/2016/03/complete-guide-parameter-tuning-xgboost-with-codes-python/