问题描述
问题陈述:
我正在Jupyter Notebook中编写一个程序,该程序动态地编写另一个脚本(script.py)。编写script.py之后,写文件的函数通过import语句运行它,然后从script.py调用一个函数。
我需要在script.py中使用熊猫,并将其导入script.py的顶部。我在script.py顶部执行NameError: name 'pd' is not defined
之后就得到了import pandas as pd
。我最初尝试省略导入语句,因为它已在调用程序中执行,但是出现了同样的错误。我尝试将import语句放入script.py的函数中,但遇到相同的错误。
Update2,已解决:
该代码现在可以工作了。我敢肯定,我唯一要做的就是走开然后返回并输入%debug
,然后重新启动内核并运行所有单元。它没有找到要调试的回溯。我猜你可以说这很神奇,但是也许是重新启动内核。魔术对我来说更有意义,哈哈。
更新1: 原始示例代码实际上并未重现该错误。如果我要对其进行测试运行,则最好在实际代码中隔离该问题。我的错。我仍然无法解决该问题,但是似乎循环中有些东西构成了混乱的写语句。因为一次运行类似的代码而不循环就可以了。
这是我的真实代码:
import os
import pandas as pd
def read_files_in_folder(fp_list,path=None,arg_list=None):
'''Reads a folder of csv tables into a dictionary of dataframes.
Does this dynamically by writing a script to a file,importing the script,and running a function from the script.
Parameters:
fp_list is [str]: list of filenames or filepaths of csv files.
path is str: (optional) filepath str filenames. os.curdir if None.
arg_list is [str]: (optional) list of pd.read_csv() arguments to pass.
Returns:
df_dict is {pd.DataFrame}: dict of dataframes created from csv files.'''
df_dict = {}
if path is None:
path = os.curdir
if arg_list is None:
for fp in fp_list:
fp_var_name = fp.split('/')[-1].split('.')[0]
df_dict[fp_var_name] = pd.read_csv(path + fp)
else:
args = ''
for arg in arg_list:
args += ',' + arg
with open('script.py','w') as file:
file.write("""
import pandas as pd
def csvs_to_df_dict():
\tdf_dict = {}
""")
for fp in fp_list:
fp_var_name = fp.split('/')[-1].split('.')[0]
statement = "\tdf_dict['" + fp_var_name + "'] = pd.read_csv('" + path + fp + "'" + args + ")\n"
file.write(statement)
file.write('\treturn df_dict')
import script
df_dict = script.csvs_to_df_dict()
return df_dict
然后执行:
csv_path = os.curdir + '/csv_tables/'
filename_list = os.listdir(path=csv_path)
df_dict = read_files_in_folder(fp_list=filename_list,path=csv_path,arg_list=['index_col=0','skip_blank_lines=False'])
df_dict['abscorrup_idea.csv']
这写了script.py:
import pandas as pd
def csvs_to_df_dict():
df_dict = {}
df_dict['abscorrup_idea'] = pd.read_csv('./csv_tables/abscorrup_idea.csv',index_col=0,skip_blank_lines=False)
# ... ... ...
df_dict['sorigeq_idea'] = pd.read_csv('./csv_tables/sorigeq_idea.csv',skip_blank_lines=False)
return df_dict
但是,一旦在script.py的NameError: name 'pd' is not defined
之后从df_dict = script.csvs_to_df_dict()
输入script.py,它将返回import pandas as pd
。有关完整的错误输出,请参见下文。
如果您不通过arg_list,因此不首先创建script.py文件,它将起作用。因此,它可以立即使用,但是我想了解为什么它不能以其他方式起作用。
我最初尝试将script.py编写为一系列语句而不是函数。我以为它就像在将代码块插入到调用它的代码中一样运行,但是我无法从一个脚本调用另一个脚本的df_dict
。不同的名称空间?所以,我正在尝试一个函数。
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-26-13999e7ca3af> in <module>
----> 1 df_dict = read_files_in_folder(fp_list=filename_list,2 arg_list=['index_col=0','skip_blank_lines=False'])
<ipython-input-25-4f1e04e89145> in read_files_in_folder(fp_list,path,arg_list)
35 file.write('\treturn df_dict')
36 import script
---> 37 df_dict = script.csvs_to_df_dict()
38
39 return df_dict
~\OneDrive\Education\WGU\C749_intro_to_data_science\Module_3_Investigate_A_Dataset\Project\script.py in csvs_to_df_dict()
1
2 import pandas as pd
----> 3
4 def csvs_to_df_dict():
5 df_dict = {}
NameError: name 'pd' is not defined
更新前的原始示例,已清理并正常运行:
例如:
# script1.py #
import pandas as pd
# The following is actually part of a function
# that is called later in the same script1,# but I'm keeping it simple for the example.
df_dict = {}
with open('script2.py','w') as file:
file.write("""
# script2.py #
import pandas as pd
def run_it():
\tdf_dict = {}
""")
path = './csv_tables/'
fn = 'abscorrup_idea.csv'
file.write("\tdf_dict['abscorrup_idea'] = pd.read_csv('" + path + fn + "',skip_blank_lines=False)\n")
file.write('\treturn df_dict')
import script2
df_dict = script2.run_it()
df_dict
# script2.py #
import pandas as pd
def run_it():
df_dict = {}
df_dict['abscorrup_idea'] = pd.read_csv('./csv_tables/abscorrup_idea.csv',skip_blank_lines=False)
return df_dict
解决方法
我尝试重现您的错误,但失败了。当我只复制粘贴您的代码时,我得到一个SyntaxError
,因为您的转义有问题。但这
with open('script2.py','w') as file:
file.write("""
# script2.py #
import pandas as pd
def run_it():
df_dict = {}
df_dict["test"] = pd.DataFrame(data={"test":[1,2,3]})
return df_dict
""")
import script2
df_dict = script2.run_it()
df_dict["test"]
在我的机器上工作正常。请注意,由于我没有您的csv文件,因此我不得不采用另一个示例dataframe
。
如帖子更新所示,以下代码有效。重新启动内核似乎可以解决问题。那还是魔术。
with self.engine.begin() as conn:
conn.execute(sql_query)
conn.execute("COMMIT;")