R：仅导入和合并特定列

问题描述

我有一个来自网络的CSV URL列表，并将它们合并到一个向量中。 现在，我想用read_csv阅读此列表。

示例：

files <- c("csv_link1.csv","csv_link2.csv","csv_link3.csv",and so on....) 
data <- map_dfr(files,read_csv)

这没问题。问题是在CSV文件中，有一些列用不同的值填充。因此，例如，在CSV1中有“ V1”列，该列中填充有双精度字；在CSV中，同一列是“ V1”列，其中填充了字符。合并CSV无效，因为它们是不同的数据类型。

就我而言，我认为有两种解决方案。

1. 我只导入某些列，所以我说read_csv仅读取列（V2和V3），而不读取V1

或

1. 我使用col_types将列合并为相同的数据类型

我都尝试过，但是由于语法正确而失败。

我尝试过类似的事情

data <- map_dfr(files,read_csv(cols_only(the col names)))

但是，这不起作用。

如何仅导入和合并特定列？

在我的情况下为具体示例：

library(data.table)
library(readr)
library(purrr)


files <- c("https://www.football-data.co.uk/mmz4281/1920/EC.csv","https://www.football-data.co.uk/mmz4281/1819/EC.csv","https://www.football-data.co.uk/mmz4281/1718/EC.csv","https://www.football-data.co.uk/mmz4281/1617/EC.csv","https://www.football-data.co.uk/mmz4281/1516/EC.csv","https://www.football-data.co.uk/mmz4281/1415/EC.csv","https://www.football-data.co.uk/mmz4281/1314/EC.csv","https://www.football-data.co.uk/mmz4281/1213/EC.csv","https://www.football-data.co.uk/mmz4281/1112/EC.csv","https://www.football-data.co.uk/mmz4281/1011/EC.csv")


data <- map_dfr(files,read_csv)

Error: Can't combine `BbAH` <character> and `BbAH` <double>.

因此，列BbAH具有不同的数据类型。但我不需要此专栏。 如果由于这种不同的数据类型问题，我可以选择合并之前合并的列，那将很酷。

解决方法

阅读csvs之后，我们可以select所需的列，并使用map_df将它们合并。

library(tidyverse)
result <- map_df(files,~read_csv(.x) %>% select(Date,HomeTeam,AwayTeam,FTHG,FTAG,FTR))

,

由于只需要这七个变量，因此可以使用fread读取这些特定的变量，以避免BbAH变量出现问题。

library(data.table)
library(dplyr)
library(purrr)

files <- c("https://www.football-data.co.uk/mmz4281/1920/EC.csv","https://www.football-data.co.uk/mmz4281/1819/EC.csv","https://www.football-data.co.uk/mmz4281/1718/EC.csv","https://www.football-data.co.uk/mmz4281/1617/EC.csv","https://www.football-data.co.uk/mmz4281/1516/EC.csv","https://www.football-data.co.uk/mmz4281/1415/EC.csv","https://www.football-data.co.uk/mmz4281/1314/EC.csv","https://www.football-data.co.uk/mmz4281/1213/EC.csv","https://www.football-data.co.uk/mmz4281/1112/EC.csv","https://www.football-data.co.uk/mmz4281/1011/EC.csv")

# Identify columns you need
myColumns = c("Date","Time","HomeTeam","AwayTeam","FTHG","FTAG","FTR")

# Modified function found in https://stackoverflow.com/a/51348578/8535855
# takes a filename and a vector of columns as input
fread_allfiles <- function(file,columns){
  x <- fread(file,select = columns) %>% 
    select(everything())   # 
  return(x)
}

df_all <- files %>% 
  map_df(~ fread_allfiles(.,myColumns))

head(df_all)

产生以下格式：

         Date  Time     HomeTeam       AwayTeam FTHG FTAG FTR
1: 03/08/2019 12:30    Stockport     Maidenhead    0    1   A
2: 03/08/2019 15:00    Aldershot          Fylde    1    2   A
3: 03/08/2019 15:00       Barnet         Yeovil    1    0   H
4: 03/08/2019 15:00 Chesterfield Dover Athletic    1    2   A
5: 03/08/2019 15:00      Chorley        Bromley    0    0   D
6: 03/08/2019 15:00  Dag and Red         Woking    0    2   A

然后可以根据需要重新格式化Date和Time列。看起来第一个文件上有Time的任何值吗？因此，其余部分将以NA

> str(df_all)
Classes ‘data.table’ and 'data.frame':  5429 obs. of  7 variables:
 $ Date    : chr  "03/08/2019" "03/08/2019" "03/08/2019" "03/08/2019" ...
 $ Time    : chr  "12:30" "15:00" "15:00" "15:00" ...
 $ HomeTeam: chr  "Stockport" "Aldershot" "Barnet" "Chesterfield" ...
 $ AwayTeam: chr  "Maidenhead" "Fylde" "Yeovil" "Dover Athletic" ...
 $ FTHG    : int  0 1 1 1 0 0 1 1 2 1 ...
 $ FTAG    : int  1 2 0 2 0 2 0 4 2 3 ...
 $ FTR     : chr  "A" "A" "H" "A" ...
 - attr(*,".internal.selfref")=<externalptr>

,

如何？

library(data.table)
library(readr)

rbindlist(lapply(files,read_csv,col_types = "character"))

这会将所有列导入为character，因此您需要在合并后将它们转换为您最初想要的样子。