问题描述
我有3列(DM1_ID,DM2_ID,对)熊猫数据帧,具有100万条记录。此外,我还有一个包含键和多个值的字典。 该函数检查字典值并获取键并将该键放在new_ID字段中。 该功能对于熊猫数据框的一小部分工作正常,但是当我将其应用于整个数据框时,会出现“ stopiteration”错误。
DM1_ID DM2_ID pairs
86503 11945.0 11945.0 [11945.0,11945.0]
86504 11945.0 362380.0 [11945.0,362380.0]
86505 11945.0 538395.0 [11945.0,538395.0]
86506 538395.0 591587.0 [11945.0,591587.0]
86507 11946.0 11946.0 [11946.0,11946.0]
86508 362380.0 200589 [362380.0,200589.0]
86509 564785.0 11946.0 [564785.0,11946.0]
f = lambda x: next(k for k,v in jdic.items() if any(i in v for i in x))
jdic = {10045: [1,6,7,10045,15,45,55,80],11945: [11945,362380,20589,10,27,538395,591587],3: [3,21,28,32,35],11946: [11946,39,564785]}
largeFile13000['new_ID'] = largeFile13000['pairs'].apply(f)
largeFile13000.drop('pairs',axis=1,inplace=True)
largeFile13000.head()
# final result I'm expecting is
DM1_ID DM2_ID new_ID
86503 11945.0 11945.0 11945
86504 11945.0 362380.0 11945
86505 11945.0 538395.0 11945
86506 538395.0 591587.0 11945
86507 11946.0 11946.0 11946
86508 362380.0 200589 11945
86509 564785.0 11946.0 11946
# error
stopiteration Traceback (most recent call last)
<ipython-input-14-ddbcd19d6baa> in <module>
----> 1 largeFile13000['new_ID'] = largeFile13000['pairs'].apply(f)
2 largeFile13000.drop('pairs',inplace=True)
3 largeFile13000.head()
c:\users\ravindu\appdata\local\programs\python\python37\lib\site-packages\pandas\core\series.py in apply(self,func,convert_dtype,args,**kwds)
3589 else:
3590 values = self.astype(object).values
-> 3591 mapped = lib.map_infer(values,f,convert=convert_dtype)
3592
3593 if len(mapped) and isinstance(mapped[0],Series):
pandas\_libs\lib.pyx in pandas._libs.lib.map_infer()
<ipython-input-12-b4ce01c34c30> in <lambda>(x)
----> 1 f = lambda x: next(k for k,v in jdic.items() if any(i in v for i in x))
stopiteration:
有人可以帮助我解决此问题吗? 预先感谢。
解决方法
从您的数据来看,基本上您只需要查找一列,例如“ DM1_ID”,因为对应的“ DM2_ID”应该属于jdic中的相同键。在这种情况下,这很容易做到。我只想把你的字典翻过来。
jdic = {10045: [1,6,7,10045,15,45,55,80],11945: [11945,362380,20589,10,27,538395,591587],3: [3,21,28,32,35],11946: [11946,39,564785]}
ndic = {}
for key in jdic:
for i in jdic[key]:
ndic[i] = key
然后应用条件。
largeFile13000['new_ID'] = largeFile13000['DM1_ID'].apply(lambda x: ndic[x])
顺便说一句,我不知道您是否有任何特定的理由以这种方式构造字典jdic。对于这种显然多对一的关系,最好使用“许多”一侧作为键。