问题描述
总结
我创建了一个扩展 sklearn's NMF 的子类。如果我之前没有重新定义该类,则对其进行酸洗并使用 dill 取消酸洗实例会失败。
最小(非)工作示例
首先,创建一些自定义的类和实例,并pickle它们:
# imports
import dill
from sklearn.decomposition import NMF
# custom classes
class NMFwrapper(NMF): # custom child class
def fit(self,X,y):
print("Slightly modifying NMF")
self = super().fit(X=X,y=y)
return self
class CustomClass(): # dummy class
def onlymethod(self):
print("I'm the only method")
class CustomChild(CustomClass): # dummy child class
def second_method(self):
print("I'm a new child method")
# instances
normal_nmf = NMF()
new_nmf = NMFwrapper()
custom = CustomClass()
child = CustomChild()
# pickle all of them
with open("nmf.pkl","wb") as f:
dill.dump(normal_nmf,f)
with open("nmf_wrapper.pkl","wb") as f:
dill.dump(new_nmf,f)
with open("custom.pkl","wb") as f:
dill.dump(custom,f)
with open("child.pkl","wb") as f:
dill.dump(child,f)
然后在第二个终端/内核/REPL/...中执行以下操作:
import dill
with open("nmf.pkl","rb") as f:
normal_nmf = dill.load(f) # works
with open("custom.pkl","rb") as f:
custom = dill.load(f) # works
with open("child.pkl","rb") as f:
child = dill.load(f) # works
with open("nmf_wrapper.pkl","rb") as f:
new_nmf = dill.load(f) # FAILS
问题
最后一次加载失败,堆栈跟踪如下:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-2-2327d8f76861> in <module>
7 child = dill.load(f)
8 with open("nmf_wrapper.pkl","rb") as f:
----> 9 new_nmf = dill.load(f)
~/miniconda3/envs/test/lib/python3.9/site-packages/dill/_dill.py in load(file,ignore,**kwds)
276 def load(file,ignore=None,**kwds):
277 """unpickle an object from a file"""
--> 278 return Unpickler(file,ignore=ignore,**kwds).load()
279
280 def loads(str,**kwds):
~/miniconda3/envs/test/lib/python3.9/site-packages/dill/_dill.py in load(self)
479
480 def load(self): #NOTE: if settings change,need to update attributes
--> 481 obj = StockUnpickler.load(self)
482 if type(obj).__module__ == getattr(_main_module,'__name__','__main__'):
483 if not self._ignore:
~/miniconda3/envs/test/lib/python3.9/site-packages/dill/_dill.py in find_class(self,module,name)
469 return type(None) #XXX: special case: nonetype missing
470 if module == 'dill.dill': module = 'dill._dill'
--> 471 return StockUnpickler.find_class(self,name)
472
473 def __init__(self,*args,**kwds):
AttributeError: Can't get attribute 'NMFwrapper' on <module '__main__'>
变化
用 dill
替换 pickle
更糟糕:只有第一次加载(正常 NMF)有效,其他三个都失败。
重新定义自定义 Wrapper(用于 dill)和自定义类/子类(用于 pickle)之前 unpickling 工作正常。
问题
为什么 dill/pickle 可以反序列化一个 NMF 实例,但在自定义类/子类上有各自的麻烦?能够反序列化自定义类和自定义类的子类,而不是“普通”类的子类,这很奇怪。
解决方法
暂无找到可以解决该程序问题的有效方法,小编努力寻找整理中!
如果你已经找到好的解决方法,欢迎将解决方案带上本链接一起发送给小编。
小编邮箱:dio#foxmail.com (将#修改为@)