在 zeppelin notebook 中运行 spark 作业时遇到问题

问题描述

过去一周我一直在尝试调试它，但没有成功。起初我认为这是一个序列化错误，因为对 readSerialData 进行了重复调用，但是即使我尝试对 DataFrame 进行操作并只返回一个向量（首先使用 mapPartition，然后只使用一个map 即使它效率低下），我仍然遇到同样的错误。

这是以下堆栈跟踪：

java.lang.classNotFoundException: org.apache.zeppelin.spark.SparkZeppelinContext
    at java.lang.classLoader.findClass(ClassLoader.java:524)
    at org.apache.spark.util.ParentClassLoader.findClass(ParentClassLoader.scala:26)
    at java.lang.classLoader.loadClass(ClassLoader.java:419)
    at org.apache.spark.util.ParentClassLoader.loadClass(ParentClassLoader.scala:34)
    at java.lang.classLoader.loadClass(ClassLoader.java:352)
    at org.apache.spark.util.ParentClassLoader.loadClass(ParentClassLoader.scala:30)
    at org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:82)
    at java.lang.classLoader.loadClass(ClassLoader.java:419)
    at java.lang.classLoader.loadClass(ClassLoader.java:352)
    at java.lang.class.getDeclaredFields0(Native Method)
    at java.lang.class.privateGetDeclaredFields(Class.java:2583)
    at java.lang.class.getDeclaredField(Class.java:2068)
    at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1857)
    at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:79)
    at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:506)
    at java.io.ObjectStreamClass$3.run(ObjectStreamClass.java:494)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:494)
    at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:391)
    at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:681)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1939)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1805)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2096)
    at java.io.ObjectInputStream.readobject0(ObjectInputStream.java:1624)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2341)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2265)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2123)
    at java.io.ObjectInputStream.readobject0(ObjectInputStream.java:1624)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2341)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2265)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2123)
    at java.io.ObjectInputStream.readobject0(ObjectInputStream.java:1624)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2341)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2265)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2123)
    at java.io.ObjectInputStream.readobject0(ObjectInputStream.java:1624)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2341)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2265)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2123)

有没有人遇到过这种情况？非常感谢任何有关如何开始调试的指示。

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）