Python Mlens Ensemble：KeyError：“[Int64Index([... dtype='int64', length=105)] 都不在 [columns]”中

问题描述

以下是我收到此错误的小版本代码：KeyError: "None of [Int64Index([...],dtype='int64')] are in the [columns]"

'...' 是一系列数字，似乎与我的 X 和 y 数据帧的索引相匹配。

我正在使用 Mlens 包在一个非常大的数据集上使用 SuperLearner 进行建模（因此可扩展性很重要）。我的目标是使用数据帧结构而不是 Numpy 数组。这将解决下游问题。

到目前为止，我已经浏览了 this 和其他相关帖子，但这些解决方案似乎不适用于这里。

数据集是此处以 .csv 格式找到的 Iris 数据集：

请注意，自定义随机森林函数效果很好。但是 mlens/SuperLearner 错误。

from sklearn.ensemble import ExtraTreesClassifier,RandomForestClassifier
from sklearn.feature_selection import SelectFromModel
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from mlens.ensemble.super_learner import SuperLearner
import numpy as np
import pandas as pd

df = pd.read_csv("/home/marktest/iris_csv.csv")
type(df)
N_FOLDS = 5
RF_ESTIMATORS = 100
RANDOM_STATE = 42
class RFBasedFeatureSelector(BaseEstimator):
  
    def __init__(self,n_estimators):
        self.n_estimators = n_estimators
        self.selector = None

    def fit(self,X,y):
        clf = RandomForestClassifier(n_estimators=self.n_estimators,random_state = RANDOM_STATE,class_weight = 'balanced')
        clf = clf.fit(X,y)
        self.selector = SelectFromModel(clf,prefit=True,threshold = 0.01)

    def transform(self,X):
        if self.selector is None:
            raise AttributeError('The selector attribute has not been assigned. You cannot call transform before first calling fit or fit_transform.')
        return self.selector.transform(X)

    def fit_transform(self,y):
        self.fit(X,y)
        return self.transform(X)
df.head()
X = df.iloc[:,0:3]                                    # split off features into new dataframe
y = df.iloc[:,4]                                     # split off outcome into new dataframe

X,X_val,y,y_val = train_test_split(X,test_size=.3,random_state=RANDOM_STATE,stratify=y)
from mlens.metrics import make_scorer
from sklearn.metrics import roc_auc_score,balanced_accuracy_score
accuracy_scorer = make_scorer(roc_auc_score,average='micro',greater_is_better=True)

clf = RandomForestClassifier(RF_ESTIMATORS,class_weight='balanced')
scaler = StandardScaler()
feature_selector = RFBasedFeatureSelector(RF_ESTIMATORS)
clf.fit(feature_selector.fit_transform(scaler.fit_transform(X),y),y)
accuracy_score(y_val,clf.predict(feature_selector.transform(scaler.transform(X_val))))

ensemble = SuperLearner(folds=N_FOLDS,shuffle=True,scorer=balanced_accuracy_score,backend="threading")

preprocessing = {'pipeline-1': [StandardScaler(),RFBasedFeatureSelector(RF_ESTIMATORS)]
                 
                }

estimators = {'pipeline-1': [RandomForestClassifier(RF_ESTIMATORS,class_weight='balanced'),]
                 }

ensemble.add(estimators,preprocessing)

ensemble.add_Meta(LogisticRegression(solver='liblinear',class_weight = 'balanced'))
ensemble.fit(X,y)```

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）