问题描述
我想找到在大型矩阵上计算欧几里得距离的最内存和时间有效方法是什么。我在下面比较了一些我知道的软件包:paralleldist
、geodist
、fields
和 stats
。我还考虑了结合 Rcpp
和 bigmemory
的 customized function 。以下是我发现的结果(下面的 reprex),但我想知道是否有其他有效的 pacakges/解决方案来完成此任务:
结果
benchmrk
#> package time alloc
#>1: pardist 0.298 5.369186e-04
#>2: fields 1.079 9.486198e-03
#>3: rcpp 54.422 2.161113e+00
#>4: stats 0.770 5.788603e+01
#>5: geodist 2.513 1.157635e+02
# plot
ggplot(benchmrk,aes(x=alloc,y=time,color= package,label=package)) +
geom_label(alpha=.5) +
coord_trans(x="log10",y="log10") +
theme(legend.position = "none")
Reprex
library(paralleldist)
library(geodist)
library(fields)
library(stats)
library(bigmemory)
library(Rcpp)
library(lineprof)
library(geobr)
library(sf)
library(ggplot2)
library(data.table)
# data input
df <- geobr::read_weighting_area()
gc(reset = T)
# convert projection to UTM
df <- st_transform(df,crs = 3857)
# get spatial coordinates
coords <- suppressWarnings(st_coordinates( st_centroid(df) ))
# prepare customized rcpp function
sourceCpp("euc_dist.cpp")
bigMatrixEuc <- function(bigMat){
zeros <- big.matrix(nrow = nrow(bigMat)-1,ncol = nrow(bigMat)-1,init = 0,type = typeof(bigMat))
BigArmaEuc(bigMat@address,zeros@address)
return(zeros)
}
### Start tests
perf_fields <- lineprof(dist_fields <- fields::rdist(coords) )
perf_geodist <- lineprof(dist_geodist <- geodist::geodist(coords,measure = "cheap") )
perf_stats <- lineprof(dist_stats <- stats::dist(coords) )
perf_pardist <- lineprof(dist_pardist <- paralleldist::pardist(coords,method = "euclidean") )
perf_rcpp <- lineprof(dist_rcpp <- bigMatrixEuc( as.big.matrix(coords) ) )
perf_fields$package <- 'fields'
perf_geodist$package <- 'geodist'
perf_stats$package <- 'stats'
perf_pardist$package <- 'pardist'
perf_rcpp$package <- 'rcpp'
# gather results
benchmrk <- rbind(perf_fields,perf_geodist,perf_stats,perf_pardist,perf_rcpp)
benchmrk <- setDT(benchmrk)[,.(time =sum(time),alloc = sum(alloc)),by=package][order(alloc)]
benchmrk
解决方法
暂无找到可以解决该程序问题的有效方法,小编努力寻找整理中!
如果你已经找到好的解决方法,欢迎将解决方案带上本链接一起发送给小编。
小编邮箱:dio#foxmail.com (将#修改为@)