解析R中的URL以提取特定数据

问题描述

我有一个数据文件，其中有一个URL列。看起来像这样 "https://www.google.com/ | query_string=utm_source=abc&utm_medium=yts&utm_campaign=123campaign&utm_term=camp%123&utm_content=brand&gclid=abcdefg|user_agent=xyz"

我希望这些数据放在单独的列中，其各自的值如下所示

utm_source utm_medium utm_campaign utm_term utm_content user_agent
abc          yts      123campaign  camp%123   brand         xyz

使用dput作为URL结果

c("https://www.google.com/ | query_string=null | ip_address=123.113.64.211 | user_agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/85.0.4183.102 Safari/537.36","https://www.google.com/ | query_string=gclid=Lxi6sNo-A17RohDAcQgvD_fw4 | ip_address=167.11.116.237 | user_agent=Mozilla/5.0 (Linux; Android 8.0.0; SM-C701F) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/70.0.3538.110 Mobile Safari/537.36","http://m.facebook.com/ | query_string=utm_source=fb&utm_medium=ctw&utm_campaign=abcPant_rem&utm_content=PantShirt | ip_address=106.193.181.252 | user_agent=Mozilla/5.0 (Linux; Android 10; SM-G975F Build/QP1A.190711.020; wv) AppleWebKit/537.36 (KHTML,like Gecko) Version/4.0 Chrome/81.0.4044.138 Mobile Safari/537.36 [FB_IAB/FB4A;FBAV/218.0.0.32.158;]")

解决方法

URL中只有一个条目在查询字符串中包含多个字段，而第一个不包含任何字段。您实际上不能从问题中的示例制作数据框架，但可以创建一个包含查询字符串中字段的命名矢量列表，如下所示：

queries <- sapply(strsplit(sapply(strsplit(URL,"query_string="),`[`,2)," \\|"),1)

lapply(strsplit(queries,"\\&|="),function(x) 
  setNames(x[seq(length(x)/2) * 2],x[seq(length(x)/2) * 2 - 1]))
#> [[1]]
#> null 
#>   NA 
#> 
#> [[2]]
#>                       gclid 
#> "Lxi6sNo-A17RohDAcQgvD_fw4" 
#> 
#> [[3]]
#>    utm_source    utm_medium  utm_campaign   utm_content 
#>          "fb"         "ctw" "abcPant_rem"   "PantShirt" 

,

这是使用提供的URL的正则表达式解决方案。

url <- "https://www.google.com/ | query_string=utm_source=abc&utm_medium=yts&utm_campaign=123campaign&utm_term=camp%123&utm_content=brand&gclid=abcdefg|user_agent=xyz"

str_match_all提取模式。

• \\w+：匹配一个或多个单词字符
• (...)：捕获组
• (?:...)?：将组匹配零次或一次，但不捕获该组。这用于处理URL的query_string=部分。

stringr::str_match_all(url,"(?:\\w+=)?(\\w+)=(\\w+)")
#> [[1]]
#>      [,1]                          [,2]           [,3]         
#> [1,] "query_string=utm_source=abc" "utm_source"   "abc"        
#> [2,] "utm_medium=yts"              "utm_medium"   "yts"        
#> [3,] "utm_campaign=123campaign"    "utm_campaign" "123campaign"
#> [4,] "utm_term=camp"               "utm_term"     "camp"       
#> [5,] "utm_content=brand"           "utm_content"  "brand"      
#> [6,] "gclid=abcdefg"               "gclid"        "abcdefg"    
#> [7,] "user_agent=xyz"              "user_agent"   "xyz"      

str_match_all返回矩阵列表，其中第一列是完全匹配项，后跟每个捕获的组。仅保留捕获的组。

stringr::str_match_all(url,"(?:\\w+=)?(\\w+)=(\\w+)")[[1]][,2:3]
#>      [,1]           [,2]         
#> [1,] "utm_source"   "abc"        
#> [2,] "utm_medium"   "yts"        
#> [3,] "utm_campaign" "123campaign"
#> [4,] "utm_term"     "camp"       
#> [5,] "utm_content"  "brand"      
#> [6,] "gclid"        "abcdefg"    
#> [7,] "user_agent"   "xyz"