问题描述
第一篇文章！ 我正在尝试使用带有HADOOP和Yarn的mapreduce运行WordCount程序，但出现此错误：
线程“ main”中的异常java.lang.NoClassDefFoundError：org / apache / hadoop / yarn / exceptions / YarnRuntimeException
我正在按照此网站上的说明进行操作： https://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html#Example:_WordCount_v1.0
解决方法
warning: You have loaded library /usr/local/hadoop/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>',or link it with '-z noexecstack'.
2020-08-16 17:49:11,518 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2020-08-16 17:49:18,923 INFO client.RMProxy: Connecting to ResourceManager at Pi1-Master/192.168.0.101:8050
2020-08-16 17:49:22,858 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2020-08-16 17:49:24,009 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/pi/.staging/job_1597607894992_0005
2020-08-16 17:49:24,947 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false,remoteHostTrusted = false
2020-08-16 17:49:26,412 INFO input.FileInputFormat: Total input files to process : 2
2020-08-16 17:49:26,674 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false,907 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false,remoteHostTrusted = false
2020-08-16 17:49:27,036 INFO mapreduce.JobSubmitter: number of splits:2
2020-08-16 17:49:28,028 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false,remoteHostTrusted = false
2020-08-16 17:49:29,257 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1597607894992_0005
2020-08-16 17:49:29,258 INFO mapreduce.JobSubmitter: Executing with tokens: []
2020-08-16 17:49:31,061 INFO conf.Configuration: resource-types.xml not found
2020-08-16 17:49:31,066 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2020-08-16 17:49:31,670 INFO impl.YarnClientImpl: Submitted application application_1597607894992_0005
2020-08-16 17:49:32,115 INFO mapreduce.Job: The url to track the job: http://Pi1-Master:8088/proxy/application_1597607894992_0005/
2020-08-16 17:49:32,122 INFO mapreduce.Job: Running job: job_1597607894992_0005
2020-08-16 17:50:03,467 INFO mapreduce.Job: Job job_1597607894992_0005 running in uber mode : false
2020-08-16 17:50:03,479 INFO mapreduce.Job: map 0% reduce 0%
2020-08-16 17:50:03,622 INFO mapreduce.Job: Job job_1597607894992_0005 failed with state FAILED due to: Application application_1597607894992_0005 failed 2 times due to AM Container for appattempt_1597607894992_0005_000002 exited with exitCode: 1
Failing this attempt.Diagnostics: [2020-08-16 16:50:17.244]Exception from container-launch.
Container id: container_1597607894992_0005_02_000001
Exit code: 1
[2020-08-16 16:50:17.282]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: A JNI error has occurred,please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/exceptions/YarnRuntimeException
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
[2020-08-16 16:50:17.284]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
Error: A JNI error has occurred,please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/exceptions/YarnRuntimeException
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
For more detailed output,check the application tracking page: http://Pi1-Master:8088/cluster/app/application_1597607894992_0005 Then click on links to logs of each attempt.
. Failing the application.
2020-08-16 17:50:03,916 INFO mapreduce.Job: Counters: 0