我有一个代码,我需要设置三个表.为此,我需要为每个表调用jdbc函数三次.见下面的代码
val props = new Properties props.setProperty("user","root") props.setProperty("password","pass") val df0 = sqlContext.read.jdbc( "jdbc:MysqL://127.0.0.1:3306/Firm42","company",props) val df1 = sqlContext.read.jdbc( "jdbc:MysqL://127.0.0.1:3306/Firm42","employee",props) val df2 = sqlContext.read.jdbc( "jdbc:MysqL://127.0.0.1:3306/Firm42","company_employee",props) df0.registerTempTable("company") df1.registerTempTable("employee") df2.registerTempTable("company_employee") val rdf = sqlContext.sql( """some_sql_query_with_joins_of_varIoUs_tables""".stripMargin) rdf.show
解决方法
DRY:
val url = "jdbc:MysqL://127.0.0.1:3306/Firm42" val tables = List("company","company_employee") val dfs = for { table <- tables } yield (table,sqlContext.read.jdbc(url,table,props)) for { (name,df) <- dfs } df.registerTempTable(name)
不需要数据框架?跳过第一个循环:
for { table <- tables } sqlContext.read.jdbc(url,props).registerTempTable(table)