问题描述
我一直在尝试为正在构建的脚本的 sql 查询中的“colname”参数应用 f 字符串格式,但我一直收到解析异常错误。 希望你能帮助我。
def expect_primary_key_have_relevant_foreign_key(spark_df1,spark_df2,colname):
'''
Check that all the primary keys have a relevant foreign key
'''
# Create Temporary View
spark_df1.createOrReplaceTempView("spark_df1")
spark_df2.createOrReplaceTempView("spark_df2")
# Wrap Query in spark.sql
result = spark.sql("""
select df1.*
from spark_df1 df1
left join
spark_df2 df2
f"on trim(upper(df1.{colname})) = trim(upper(df2.{colname}))"
f"where df2.{colname} is null"
""")
if result == 0:
print("Validation Passed!")
else:
print("Validation Failed!")
return result
非常感谢
解决方法
我找到了解决方案, f 在三重引号 """ 之前:
# Wrap Query in spark.sql
result = spark.sql(f"""
select df1.*
from spark_df1 df1
left join
spark_df2 df2
on trim(upper(df1.{colname})) = trim(upper(df2.{colname}))
where df2.{colname} is null
""")