基于 gluonts deepstate 模型的预测在 Linux OS 上运行时很慢

问题描述

观察到 gluonTS 深度状态时间序列模型在不同的操作系统中表现不同。在 Mac OS(四核、物理 VM、intel i7 @2.3 GHz)上运行 docker 容器时,预测需要大约 3 秒;当它在 Linux 操作系统上的 docker 容器上运行时,这个函数需要大约 8 秒(仅 4 个 cpu,VM,intel broadwell @2.19 GHz);同样的函数在 kubernetes 集群上运行时大约需要 20 秒(分配有 16 个 cpu,VM,BM.standard2.52)。在此期间监视 cpu / 内存时,可以观察到该程序试图消耗所有分配的 cpu,同时最少占用内存和页面文件。我在 Linux 环境下运行 cProfile,下面是分析器的简短输出。我知道 mxnet 带有不同的库来支持不同的操作系统。是否存在有关 mxnet 性能的已知问题?我如何能最好地确定性能问题。

        1    0.000    0.000    7.819    7.819 /usr/local/lib/python3.7/site-packages/xxxxxx/forecast/forecast_client.py:26(predict)
        1    0.000    0.000    7.801    7.801 /usr/local/lib/python3.7/site-packages/xxxxxx/forecaster.py:8(predict)
        2    0.000    0.000    7.801    3.900 /usr/local/lib/python3.7/site-packages/gluonts/mx/model/predictor.py:157(predict)
        2    0.000    0.000    7.800    3.900 /usr/local/lib/python3.7/site-packages/gluonts/model/forecast_generator.py:113(__call__)
        1    0.000    0.000    7.786    7.786 /usr/local/lib/python3.7/functools.py:835(wrapper)
        1    0.000    0.000    7.786    7.786 /usr/local/lib/python3.7/site-packages/gluonts/mx/model/predictor.py:61(_)
   5901/1    0.029    0.000    7.785    7.785 /usr/local/lib/python3.7/site-packages/mxnet/gluon/block.py:688(__call__)
   5901/1    0.075    0.000    7.785    7.785 /usr/local/lib/python3.7/site-packages/mxnet/gluon/block.py:1127(forward)
        1    0.001    0.001    7.764    7.764 /usr/local/lib/python3.7/site-packages/gluonts/model/deepstate/_network.py:216(hybrid_forward)
    89852    3.323    0.000    4.976    0.000 /usr/local/lib/python3.7/site-packages/mxnet/_ctypes/ndarray.py:80(_imperative_invoke)
        2    0.000    0.000    4.301    2.151 /usr/local/lib/python3.7/site-packages/gluonts/model/deepstate/_network.py:102(compute_lds)
        2    0.002    0.001    4.239    2.119 /usr/local/lib/python3.7/site-packages/mxnet/gluon/rnn/rnn_cell.py:803(unroll)
        4    0.010    0.003    4.226    1.057 /usr/local/lib/python3.7/site-packages/mxnet/gluon/rnn/rnn_cell.py:195(unroll)
5880/2352    0.021    0.000    4.162    0.002 /usr/local/lib/python3.7/site-packages/mxnet/gluon/rnn/rnn_cell.py:287(forward)
     2352    0.019    0.000    3.939    0.002 /usr/local/lib/python3.7/site-packages/mxnet/gluon/rnn/rnn_cell.py:958(hybrid_forward)
        1    0.001    0.001    2.847    2.847 /usr/local/lib/python3.7/site-packages/gluonts/mx/distribution/lds.py:201(log_prob)
        1    0.024    0.024    2.840    2.840 /usr/local/lib/python3.7/site-packages/gluonts/mx/distribution/lds.py:238(kalman_filter)
     2352    0.085    0.000    2.313    0.001 /usr/local/lib/python3.7/site-packages/mxnet/gluon/rnn/rnn_cell.py:527(hybrid_forward)
      672    0.041    0.000    2.290    0.003 /usr/local/lib/python3.7/site-packages/gluonts/mx/distribution/lds.py:581(kalman_filter_step)
     1176    0.007    0.000    1.383    0.001 /usr/local/lib/python3.7/site-packages/mxnet/gluon/rnn/rnn_cell.py:988(hybrid_forward)
     2352    0.019    0.000    1.092    0.000 /usr/local/lib/python3.7/site-packages/mxnet/gluon/rnn/rnn_cell.py:969(<listcomp>)
      672    0.026    0.000    0.790    0.001 /usr/local/lib/python3.7/site-packages/gluonts/mx/distribution/multivariate_gaussian.py:73(log_prob)
    89852    0.403    0.000    0.760    0.000 /usr/local/lib/python3.7/site-packages/mxnet/ndarray/register.py:75(_verify_all_legacy_ndarrays)
     4704    0.020    0.000    0.740    0.000 /usr/local/lib/python3.7/site-packages/mxnet/gluon/rnn/rnn_cell.py:961(<lambda>)
    10249    0.052    0.000    0.666    0.000 <string>:2(linalg_gemm2)

关于环境:

  • python 3.7
  • gluonts 0.6.1
  • mxnet 1.6.0

解决方法

暂无找到可以解决该程序问题的有效方法,小编努力寻找整理中!

如果你已经找到好的解决方法,欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@)

相关问答

Selenium Web驱动程序和Java。元素在(x,y)点处不可单击。其...
Python-如何使用点“。” 访问字典成员?
Java 字符串是不可变的。到底是什么意思?
Java中的“ final”关键字如何工作?(我仍然可以修改对象。...
“loop:”在Java代码中。这是什么,为什么要编译?
java.lang.ClassNotFoundException:sun.jdbc.odbc.JdbcOdbc...