在 Numpy Python 中对每一行进行基准测试

问题描述

有没有一种工具可以用来获取我的 Python 程序的基准。我想看看每行需要多长时间来处理而不是为每行使用 %timeit 是否有其他库可以帮助我实现这一目标?

代码

%%time
from numpy import random
Appending_list = []
Values = random.randint(100,size=(100000))
Number_array = random.randint(100,size=(1000))
for n in range(len(Values)):
    result = np.sum(Number_array) + Values[n] * len(Number_array)
    Appending_list.append(result)

enter image description here

解决方法

您可以使用 PySnooper 库:

您必须将代码包装在函数或类中。然后用 @pysnooper.snoop()
装饰该函数或类 根据您的要求设置 relative_time=True

这是一个例子:

import pysnooper

@pysnooper.snoop(relative_time=True)
def number_to_bits(number):
    if number:
        bits = []
        while number:
            number,remainder = divmod(number,2)
            bits.insert(0,remainder)
        return bits
    else:
        return [0]

调用函数后:number_to_bits(10)

输出:

Source path:... <ipython-input-28-59cb3f746130>
Starting var:.. number = 10
00:00:00.000002 call         4 def number_to_bits(number):
00:00:00.000752 line         5     if number:
00:00:00.000884 line         6         bits = []
New var:....... bits = []
00:00:00.000956 line         7         while number:
00:00:00.001070 line         8             number,2)
Modified var:.. number = 5
New var:....... remainder = 0
00:00:00.001146 line         9             bits.insert(0,remainder)
Modified var:.. bits = [0]
00:00:00.001299 line         7         while number:
00:00:00.001395 line         8             number,2)
Modified var:.. number = 2
Modified var:.. remainder = 1
00:00:00.001468 line         9             bits.insert(0,remainder)
Modified var:.. bits = [1,0]
00:00:00.001571 line         7         while number:
00:00:00.001653 line         8             number,2)
Modified var:.. number = 1
Modified var:.. remainder = 0
00:00:00.001715 line         9             bits.insert(0,remainder)
Modified var:.. bits = [0,1,0]
00:00:00.001842 line         7         while number:
00:00:00.001953 line         8             number,2)
Modified var:.. number = 0
Modified var:.. remainder = 1
00:00:00.002030 line         9             bits.insert(0,0]
00:00:00.002155 line         7         while number:
00:00:00.002281 line        10         return bits
00:00:00.002355 return      10         return bits
Return value:.. [1,0]
Elapsed time: 00:00:00.002523
[1,0]


,

虽然有分析工具,但我认为明智地使用 FloatBuffer::put() 通常会更好。

例如。整个循环(减去导入)

@ObservedObject

每个循环 1.19 s ± 937 µs(平均值 ± 标准偏差,7 次运行,每次 1 次循环)

但是 @StateObject 步骤是次要的

template <class OP> void function(OP op)
{
  // call with int
  op(1,2);
  // or with double
  op(1.2,2.3);
  // call with explicit template argument
  op.template operator()<int>(1,2);
  op.template operator()<string>("one","two");
}

struct Add
{
  template <class T> T operator ()(T a,T b)
  {
    return a + b;
  }
};

function(Add());
// or call with C++14 lambda
function([](auto a,auto b) { return a + b; });

但除了 append 之外,该循环仅执行 100000 次求和行。让时间独处:

timeit

但是等等,为什么要执行 In [15]: %%timeit ...: Appending_list = [] ...: Values = np.random.randint(100,size=(100000)) ...: Number_array = np.random.randint(100,size=(1000)) ...: for n in range(len(Values)): ...: result = np.sum(Number_array) + Values[n] * len(Number_array) ...: Appending_list.append(result) ...: 100000 次;只做一次

random

更好的是,一次执行 In [16]: %timeit Values = np.random.randint(100,size=(100000)) ...: 1.24 ms ± 404 ns per loop (mean ± std. dev. of 7 runs,1000 loops each) In [18]: %%timeit Values = np.random.randint(100,size=(100000)); Number_array = np.random.randint(100,si ...: ze=(1000)) ...: Appending_list = [] ...: for n in range(len(Values)): ...: result = np.sum(Number_array) + Values[n] * len(Number_array) ...: Appending_list.append(result) ...: ...: 1.19 s ± 2.4 ms per loop (mean ± std. dev. of 7 runs,1 loop each) 乘法,而不是循环:

In [19]: timeit (np.sum(Number_array)+Values*Number_array.shape[0])
313 µs ± 4.1 µs per loop (mean ± std. dev. of 7 runs,1000 loops each)

我也验证了 [21] 的计算也做了同样的事情。