Python3 functools lru_cache RuntimeError

问题描述

from functool import lru_cache


@lru_cache
def fibonacci(n):
    """0,1,2,3,5,8,13,21,34
    """

    if n == 0:
        yield 0
    elif n == 1:
        yield 1
    else:
        yield next(fibonacci(n - 1)) + next(fibonacci(n - 2))

如果我使用@lru_cache装饰器调用此函数，如下所示：

for x in range(10):
    print(next(fibonacci(x)))

我得到：

stopiteration

The above exception was the direct cause of the following exception:

RuntimeError: generator raised stopiteration

我已经做了很多搜索，但我不知道该如何解决。没有装饰器，一切正常。

解决方法

如果您确实想缓存并因此重用生成器迭代器，请确保它们确实支持该迭代器。就是说，使他们不仅产生结果，而且反复产生结果。例如：

@lru_cache
def fibonacci(n):
    """0,1,2,3,5,8,13,21,34
    """

    if n == 0:
        while True:
            yield 0
    elif n == 1:
        while True:
            yield 1
    else:
        result = next(fibonacci(n - 1)) + next(fibonacci(n - 2))
        while True:
            yield result

测试：

>>> for x in range(10):
        print(next(fibonacci(x)))

0
1
1
2
3
5
8
13
21
34

,

您可以使用备忘录装饰器

参考：Can I memoize a Python generator?的Jasmijn回答

代码

from itertools import tee
from types import GeneratorType

Tee = tee([],1)[0].__class__

def memoized(f):
    cache={}
    def ret(*args):
        if args not in cache:
            cache[args]=f(*args)
        if isinstance(cache[args],(GeneratorType,Tee)):
            # the original can't be used any more,# so we need to change the cache as well
            cache[args],r = tee(cache[args])
            return r
        return cache[args]
    return ret

@memoized
def Fibonacci(n):
    """0,34
    """

    if n == 0:
        yield 0
    elif n == 1:
        yield 1
    else:
        yield next(fibonacci_mem(n - 1)) + next(fibonacci_mem(n - 2))

计时测试

摘要

测试n从1到20 orig：原始代码 lru：使用lru缓存 记忆：使用记忆修饰器

每种算法的3次运行以秒为单位

结果表明，lru_cache技术提供了最快的运行时间（即较短的时间）

n: 1 orig: 0.000008,lru 0.000006,mem: 0.000015
n: 10 orig: 0.000521,lru 0.000024,mem: 0.000057
n: 15 orig: 0.005718,lru 0.000013,mem: 0.000035
n: 20 orig: 0.110947,lru 0.000014,mem: 0.000040
n: 25 orig: 1.503879,lru 0.000018,mem: 0.000042

计时测试代码

from itertools import tee
from types import GeneratorType
from functools import lru_cache

Tee = tee([],r = tee(cache[args])
            return r
        return cache[args]
    return ret
    
def fibonacci(n):
    """0,34
    """

    if n == 0:
        yield 0
    elif n == 1:
        yield 1
    else:
        yield next(fibonacci(n - 1)) + next(fibonacci(n - 2))

@memoized
def fibonacci_mem(n):
    """0,34
    """

    if n == 0:
        yield 0
    elif n == 1:
        yield 1
    else:
        yield next(fibonacci_mem(n - 1)) + next(fibonacci_mem(n - 2))

@lru_cache
def fibonacci_cache(n):
    """0,34
    """

    if n == 0:
        while True:
            yield 0
    elif n == 1:
        while True:
            yield 1
    else:
        result = next(fibonacci_cache(n - 1)) + next(fibonacci_cache(n - 2))
        while True:
            yield result

from timeit import timeit

cnt = 3
for n in [1,10,15,20,25]:
  t_orig = timeit(lambda:next(fibonacci(n)),number = cnt)
  t_mem = timeit(lambda:next(fibonacci_mem(n)),number = cnt)
  t_cache = timeit(lambda:next(fibonacci_cache(n)),number = cnt)
  print(f'n: {n} orig: {t_orig:.6f},lru {t_cache:.6f},mem: {t_mem:.6f}')