问题描述
我正在将一个数据帧写入 hive 表并检查该过程是否成功完成(hive 中的数据加载过程)然后我想打印消息 “数据写入过程已完成”,但如果由于某些错误而失败,即(错误的表名,错误的表名,语法错误等),则应打印 “数据写入过程失败”。我在下面写代码:
import spark.implicits._
import scala.io.source
import org.apache.spark.sql._
import scala.util.control.Exception._
import com.hortonworks.hwc.HiveWarehouseSession
import com.hortonworks.hwc.HiveWarehouseSession._
val sparksession1 = SparkSession.builder().appName("Appname")
.config("hive.exec.dynamic.partition","true")
.config("hive.exec.dynamic.partition.mode","nonstrict")
.config("hive.mapred.mode","nonstrict")
.config("spark.sql.hive.hiveserver2.jdbc.url","jdbc:hive2:xxxxxxxx")
.enableHiveSupport().getorCreate()
val Hive2 = com.hortonworks.spark.sql.hive.llap.HiveWarehouseBuilder.session(sparksession1).build()
val someDF = Seq((1,2019),(2,(3,(4,(5,(6,2019))
.toDF("MONTH","YEAR")
Hive2.setDatabase("DatabaseName")
val s= Try {
val result=someDF.write.format("com.hortonworks.spark.sql.hive.llap.HiveWarehouseConnector")
.option("database","DatabaseName").option("table","tablename").mode("append").save()
}val y = s match {
case Failure(e) => e match {
case e: Exception =>
}
case Success(s) =>
}
我收到以下错误消息
<console>:4: error: ';' expected but 'match' found.
} match {
^
scala> case Success(_) =>
<console>:1: error: illegal start of deFinition
case Success(_) =>
^
scala> println("Data Wtriting Process has been Completed")
Data Wtriting Process has been Completed
scala> case Failure(_) =>
<console>:1: error: illegal start of deFinition
case Failure(_) =>
^
scala> println("Data Wtriting Process has been Failed")
Data Wtriting Process has been Failed
scala>
| }
<console>:2: error: eof expected but '}' found.
}
^
有人可以指导我如何实现相同的。
注意:我正在使用 HDP 3.0 并在 Spark/Scala 中编写我的应用程序。
解决方法
暂无找到可以解决该程序问题的有效方法,小编努力寻找整理中!
如果你已经找到好的解决方法,欢迎将解决方案带上本链接一起发送给小编。
小编邮箱:dio#foxmail.com (将#修改为@)