问题描述
使用下面的代码和下面的 sbt,我尝试在 spark 中从数据帧写入 XML,但出现以下错误
线程“main”中的异常 java.lang.ClassNotFoundException:无法找到数据源:com.databricks.spark.xml。请在 http://spark.apache.org/third-party-projects.html
找到包裹import org.apache.spark.{SparkConf,SparkContext};
import org.apache.spark.sql.functions._
import org.apache.spark.sql.SparkSession
import com.databricks.spark.xml
object nestedutil_json
{
def main(args: Array[String])=
{
val spk1 = SparkSession.builder().appName("Spark SQL basics").getOrCreate()
import spk1.implicits._
val df_read_json = spk1.read.schema(dataSchemanstd).json("/user/pic1/dataframe_nested_amount1.json")
df_read_json.printSchema
df_read_json.write.format("com.databricks.spark.xml").option("rootTag","OfficeInfo").save("/user/pic1/xml/dataframe_nested.xml")
}
}
-----------Sbt file ------------
name := "testscala"
version := "2.1.1"
scalaVersion := "2.11.12"
libraryDependencies ++= Seq("com.typesafe.akka" %% "akka-actor" % "2.5.4","com.typesafe.akka" %%
"akka-testkit" % "2.5.4" % Test,"org.apache.spark" %% "spark-core" % "2.1.1","org.apache.spark" %%
"spark-sql" % "2.0.0","com.databricks" %% "spark-xml" % "0.3.3")
解决方法
暂无找到可以解决该程序问题的有效方法,小编努力寻找整理中!
如果你已经找到好的解决方法,欢迎将解决方案带上本链接一起发送给小编。
小编邮箱:dio#foxmail.com (将#修改为@)