您好，关于sklearn.Pipeline与时间序列的自定义转换器的两个问题

问题描述

我应该如何修改下面的代码以使其起作用：

目标，预测= pipe.fit_predict（df）

编辑：

target,predicted = pipe.fit_transform(df,df)

我的代码：

import numpy as np
import pandas as pd
from sklearn.base import BaseEstimator
from sklearn.base import TransformerMixin
from sklearn.pipeline import Pipeline 
np.random.seed(1)

rows,cols = 100,1
data = np.random.randint(100,size = (rows,cols))
tidx = pd.date_range('2019-01-01',periods=rows,freq='20min') 
df = pd.DataFrame(data,columns=['num_orders'],index=tidx)
      


class MakeFeatures(BaseEstimator,TransformerMixin):

def __init__(self,X,y = None,max_lag = None,rolling_mean_day = None,rolling_mean_month = None):
    self.X = X.resample('1H').sum()
    self.max_lag = max_lag
    self.rolling_mean_day = rolling_mean_day
    self.rolling_mean_month = rolling_mean_month
        
def fit(self,y = None):
    return self

def transform(self,y = None):
    data = pd.DataFrame(index = self.X.index)
    data['num_orders'] = self.X['num_orders']
    data['year'] = self.X.index.year
    data['month'] = self.X.index.month
    data['day'] = self.X.index.day
    data['dayofweek'] = self.X.index.dayofweek
    
    data['detrend'] = self.X.shift() - self.X
    
    if self.max_lag:
        for lag in range(1,self.max_lag + 1):
            data['lag_{}'.format(lag)] = data['detrend'].shift(lag)
    if self.rolling_mean_day:
        data['rolling_mean_24'] = data.detrend.shift().rolling(self.rolling_mean_day).mean()
    
    if self.rolling_mean_month:
        data['rolling_mean_24'] = data['detrend'].shift().rolling(self.rolling_mean_month).mean()
    
    if data['year'].mean() == data['year'][1]:
        data = data.drop('year',axis = 1)
    
    data = data.dropna()
    
    y = data.num_orders
    data = data.drop('num_orders',1)
    
    return data,y

pipe = Pipeline([
                ('features',MakeFeatures(df,df,2,24)),('scaler',StandardScaler())  
    ])

target,df)  # where ‘Target’ is y - the output from the Class

出局：

ValueError: could not broadcast input array from shape (9,7) into shape (9).

管道中的每个功能都工作正常。

我可以运行 MakeFeatures（df，df）和 StandardScaler（）。fit_transform（df，df）。

我可以将MakeFeatures（df，df）的产品插入StandardScaler，并且没有错误。

解决方法

您不能使用

目标，预测= pipe.fit_predict（df）

与定义的管道一起使用，因为fit_predict（）方法只能在估算器也实现了这种方法的情况下使用。 Reference in documentation

仅在最终估算器实现fit_predict时有效。

此外，它只会返回预测，因此您不能使用target,predicted =，而应该使用predicted =

您遇到了错误

ValueError：设置具有序列的数组元素。

因为您要提供StandardScaler()和pandas.TimeSeries。

这是因为通过方法调用pipe.fit_predict(df)，您仅向管道提供了“ X”而不是“ y”。这对于管道“ MakeFeatures”的第一个组件很好，因为它接受“ X”并返回“ data”和“ y”，但是在管道中将不使用“ y”，因为“ y”必须在fit_predict（）调用的开头定义。

在这里查看该方法的文档：https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html#sklearn.pipeline.Pipeline.fit_predict

它表示“ y”参数

培训目标。必须满足所有步骤的标签要求 管道。

因此，“ y”将用作管道所有部分的“ y”，但您的未定义，所以None。

因此，当前的管道基本上会发生以下情况：

makeF = MakeFeatures(df,2,24)
transformed_df = makeF.fit_transform(df)

sc = StandardScaler()
sc.fit(transformed_df)

并导致ValueError: setting an array element with a sequence.

所以我建议您像这样更新代码：

import numpy as np
import pandas as pd
from sklearn.base import BaseEstimator
from sklearn.base import TransformerMixin
from sklearn.pipeline import Pipeline 
from sklearn.preprocessing import StandardScaler 
from sklearn.linear_model import LinearRegression

np.random.seed(1)

rows,cols = 100,1
data = np.random.randint(100,size = (rows,cols))
tidx = pd.date_range('2019-01-01',periods=rows,freq='20min') 
df = pd.DataFrame(data,columns=['num_orders'],index=tidx)
      
class MakeFeatures(BaseEstimator,TransformerMixin):

  def __init__(self,X,max_lag = None,rolling_mean_day = None,rolling_mean_month = None):
      self.X = X.resample('1H').sum()
      self.max_lag = max_lag
      self.rolling_mean_day = rolling_mean_day
      self.rolling_mean_month = rolling_mean_month
          
  def fit(self,X):
      return self

  def transform(self,X):
      data = pd.DataFrame(index = self.X.index)
      data['num_orders'] = self.X['num_orders']
      data['year'] = self.X.index.year
      data['month'] = self.X.index.month
      data['day'] = self.X.index.day
      data['dayofweek'] = self.X.index.dayofweek
      
      data['detrend'] = self.X.shift() - self.X
      
      if self.max_lag:
          for lag in range(1,self.max_lag + 1):
              data['lag_{}'.format(lag)] = data['detrend'].shift(lag)
      if self.rolling_mean_day:
          data['rolling_mean_24'] = data.detrend.shift().rolling(self.rolling_mean_day).mean()
      
      if self.rolling_mean_month:
          data['rolling_mean_24'] = data['detrend'].shift().rolling(self.rolling_mean_month).mean()
      
      if data['year'].mean() == data['year'][1]:
          data = data.drop('year',axis = 1)
      
      data = data.dropna()
      
      y = data.num_orders
      data = data.drop('num_orders',1)
      
      return data,list(y)

pipe = Pipeline([
                 ('scaler',StandardScaler()),('Model',LinearRegression())
      ])

makeF = MakeFeatures(df,24)
makeF.fit(df)
data,y = makeF.transform(df)
pipe.fit(data,y)  # where ‘Target’ is y - the output from the Class

然后，您可以使用管道来预测数据并评估性能，例如使用r2_score：

from sklearn.metrics import r2_score

predictions = pipe.predict(data)
r2_score(y,predictions)