Spark-Solr错误，可能是日志记录或hadoop主版本

问题描述

spark-solr https://github.com/lucidworks/spark-solr

错误

此错误是当我启动$ spark-shell

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show Log4j2 internal initialization logging.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.3.1-1.cdh6.3.1.p0.1470567/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.3.1-1.cdh6.3.1.p0.1470567/lib/spark/jars/spark-solr-3.6.0-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
java.lang.IllegalArgumentException: Unrecognized Hadoop major version number: 3.0.0-cdh6.3.1
  at org.apache.hadoop.hive.shims.Shimloader.getMajorVersion(Shimloader.java:174)
  at org.apache.hadoop.hive.shims.Shimloader.loadShims(Shimloader.java:139)
  at org.apache.hadoop.hive.shims.Shimloader.getHadoopShims(Shimloader.java:100)
  at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:368)
  at org.apache.hadoop.hive.conf.HiveConf.<clinit>(HiveConf.java:105)
  at java.lang.class.forName0(Native Method)
  at java.lang.class.forName(Class.java:348)
  at org.apache.spark.util.Utils$.classForName(Utils.scala:242)
  at org.apache.spark.sql.SparkSession$.hiveClassesArePresent(SparkSession.scala:1117)
  at org.apache.spark.repl.Main$.createSparkSession(Main.scala:102)
  ... 62 elided
<console>:14: error: not found: value spark
       import spark.implicits._
              ^
<console>:14: error: not found: value spark
       import spark.sql
              ^

spark-solr中的.jar是spark-solr-3.6.0-shaded.jar和spark-solr-3.6.0.jar

我的设置是Ubuntu 18，Solr 7.4，Spark 2.4.0，Scala 2.11.12

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）