我正尝试在Windows上使用Flume（使用HDInsight预览版）。 我想使用假脱机目录将文件推送到HDFS。
这是我的configuration
agent.sources=s1 agent.channels=c1 agent.sinks=k1 agent.sources.s1.type=spooldir agent.sources.s1.spoolDir=c:Hadoop agent.channels.c1.type=memory agent.sinks.k1.type=hdfs agent.sinks.k1.channel=c1 agent.sinks.k1.hdfs.path=/Users/flu agent.sinks.k1.hdfs.inUsePrefix=inu agent.sinks.k1.hdfs.fileSuffix=.dat
这是我的脚本
"C:Hadoopjavabinjava.exe" -Xmx20m -Dlog4j.configuration=file:///%CD%conflog4j.properties - cp "C:Hadoopflume2lib*" org.apache.flume.node.Application -f C:Hadoopflume2conftest1.conf -n agent
但是我不断收到错误
apache flume hdfs可以接受dynamicpath写吗？
JSON数据下沉到Apache Phoenix与Apache Flume错误
JMeter – 找不到TestPlan类
在循环中获取“检查flume.conf中的更改”
读取Flume spoolDir并行
2013-11-28 22:31:15,341 (conf-file-poller-0) [ERROR - org.apache.flume.node.Poll ingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingProperties FileConfigurationProvider.java:145)] Failed to start agent because dependencies were not found in classpath. Error follows. java.lang.NoClassDefFoundError: org/apache/commons/configuration/Configuration at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMet ricsSystem.java:37)
我从来没有真正使用Java，所以我猜这是一个configuration问题，但我完全卡住了。 干杯。
编辑这是flume lib目录的内容
C： Hadoop flume2 lib目录
11/28/2013 10:28 PM <DIR> . 11/28/2013 10:28 PM <DIR> .. 03/27/2013 06:44 PM 346,729 apache-log4j-extras-1.1.jar 04/23/2013 09:25 PM 95,536 apache-mime4j-core-0.7.2.jar 04/23/2013 09:25 PM 304,810 apache-mime4j-dom-0.7.2.jar 02/22/2013 08:08 AM 43,033 asm-3.1.jar 03/16/2013 08:58 AM 116,219 aspectjrt-1.6.11.jar 02/22/2013 08:08 AM 16,679 async-1.3.1.jar 05/08/2013 11:15 PM 170,492 asynchbase-1.4.1.jar 02/22/2013 08:08 AM 299,377 avro-1.7.3.jar 02/22/2013 08:08 AM 186,572 avro-ipc-1.7.3.jar 04/23/2013 09:25 PM 229,116 bcmail-jdk15-1.45.jar 04/23/2013 09:25 PM 1,663,318 bcprov-jdk15-1.45.jar 04/23/2013 09:26 PM 92,027 boilerpipe-1.1.0.jar 06/23/2013 12:19 AM 22,043 cdk-morphlines-avro-0.4.0.jar 06/23/2013 12:19 AM 255,765 cdk-morphlines-core-0.4.0.jar 06/23/2013 12:19 AM 15,110 cdk-morphlines-solr-cell-0.4.0.jar 06/23/2013 12:19 AM 24,269 cdk-morphlines-solr-core-0.4.0.jar 06/23/2013 12:19 AM 7,573 cdk-morphlines-tika-core-0.4.0.jar 06/23/2013 12:19 AM 11,174 cdk-morphlines-tika-decompress-0.4.0.jar 06/23/2013 12:19 AM 7,736 cdk-morphlines-twitter-0.4.0.jar 02/22/2013 08:08 AM 41,123 commons-cli-1.2.jar 05/23/2013 09:35 AM 263,865 commons-codec-1.8.jar 02/22/2013 08:08 AM 575,389 commons-collections-3.2.1.jar 02/22/2013 08:09 AM 241,367 commons-compress-1.4.1.jar 02/22/2013 08:08 AM 160,519 commons-dbcp-1.4.jar 04/23/2013 09:25 PM 57,779 commons-fileupload-1.2.1.jar 02/22/2013 08:08 AM 163,151 commons-io-2.1.jar 02/22/2013 08:08 AM 279,193 commons-lang-2.5.jar 02/22/2013 08:08 AM 60,686 commons-logging-1.1.1.jar 02/22/2013 08:08 AM 96,221 commons-pool-1.5.4.jar 04/23/2013 09:26 PM 185,676 config-1.0.0.jar 02/22/2013 08:08 AM 2,671,577 derby-10.8.2.2.jar 03/05/2013 04:57 PM 313,898 dom4j-1.6.1.jar 06/25/2013 01:24 AM 20,430 flume-avro-source-1.4.0.jar 06/25/2013 01:23 AM 268,882 flume-file-channel-1.4.0.jar 06/25/2013 01:22 AM 62,999 flume-hdfs-sink-1.4.0.jar 06/25/2013 01:22 AM 13,532 flume-irc-sink-1.4.0.jar 06/25/2013 01:22 AM 53,389 flume-jdbc-channel-1.4.0.jar 06/25/2013 01:23 AM 25,553 flume-jms-source-1.4.0.jar 06/25/2013 01:22 AM 55,924 flume-ng-configuration-1.4.0.jar 06/25/2013 01:22 AM 339,790 flume-ng-core-1.4.0.jar 06/25/2013 01:23 AM 25,193 flume-ng-elasticsearch-sink-1.4.0.jar 06/25/2013 01:23 AM 20,266 flume-ng-embedded-agent-1.4.0.jar 06/25/2013 01:23 AM 45,705 flume-ng-hbase-sink-1.4.0.jar 06/25/2013 01:24 AM 16,413 flume-ng-log4jappender-1.4.0.jar 06/25/2013 01:23 AM 33,784 flume-ng-morphline-solr-sink-1.4.0.jar 06/25/2013 01:23 AM 31,105 flume-ng-node-1.4.0.jar 06/25/2013 01:21 AM 116,280 flume-ng-sdk-1.4.0.jar 06/25/2013 01:23 AM 44,406 flume-scribe-source-1.4.0.jar 06/25/2013 01:24 AM 56,652 flume-thrift-source-1.4.0.jar 06/25/2013 01:24 AM 15,636 flume-tools-1.4.0.jar 04/23/2013 09:25 PM 188,471 fontBox-1.7.1.jar 02/22/2013 08:08 AM 189,285 gson-2.2.2.jar 02/22/2013 08:08 AM 1,501,575 guava-10.0.1.jar 10/22/2013 01:43 PM 4,126,926 hadoop-core-1.1.0-SNAPSHOT.jar 10/22/2013 01:43 PM 4,926 hadoop-core.jar 02/22/2013 08:08 AM 427,021 httpclient-4.2.1.jar 02/22/2013 08:08 AM 223,374 httpcore-4.2.1.jar 04/23/2013 09:25 PM 26,477 httpmime-4.2.3.jar 02/22/2013 08:08 AM 132,202 irclib-1.10.jar 04/23/2013 09:26 PM 521,237 isoparser-1.0-RC-1.jar 06/19/2013 02:47 AM 33,816 jackson-annotations-2.2.1.jar 06/19/2013 02:47 AM 192,105 jackson-core-2.2.1.jar 02/22/2013 08:08 AM 228,268 jackson-core-asl-1.9.3.jar 06/19/2013 02:47 AM 864,931 jackson-databind-2.2.1.jar 02/22/2013 08:08 AM 773,019 jackson-mapper-asl-1.9.3.jar 06/19/2013 02:47 AM 17,308 jcl-over-slf4j-1.6.1.jar 03/05/2013 04:57 PM 153,253 jdom-1.0.jar 04/23/2013 09:25 PM 50,942 jempBox-1.7.1.jar 02/22/2013 08:08 AM 539,912 jetty-6.1.26.jar 02/22/2013 08:08 AM 177,131 jetty-util-6.1.26.jar 02/22/2013 08:08 AM 570,478 joda-time-2.1.jar 02/22/2013 08:08 AM 33,015 jsr305-1.3.9.jar 04/23/2013 09:26 PM 220,813 juniversalchardet-1.0.3.jar 03/27/2013 06:44 PM 301,547 libthrift-0.7.0.jar 02/22/2013 08:08 AM 481,535 log4j-1.2.16.jar 06/19/2013 02:47 AM 1,562,517 lucene-analyzers-common-4.3.0.jar 06/19/2013 02:47 AM 4,489,180 lucene-analyzers-kuromoji-4.3.0.jar 06/19/2013 02:47 AM 23,136 lucene-analyzers-phonetic-4.3.0.jar 06/19/2013 02:47 AM 296,556 lucene-codecs-4.3.0.jar 06/19/2013 02:47 AM 2,211,020 lucene-core-4.3.0.jar 06/19/2013 02:47 AM 107,729 lucene-grouping-4.3.0.jar 06/19/2013 02:47 AM 123,533 lucene-Highlighter-4.3.0.jar 06/19/2013 02:47 AM 35,901 lucene-memory-4.3.0.jar 06/19/2013 02:47 AM 91,831 lucene-misc-4.3.0.jar 06/19/2013 02:47 AM 202,541 lucene-queries-4.3.0.jar 06/19/2013 02:47 AM 385,602 lucene-queryparser-4.3.0.jar 06/19/2013 02:47 AM 92,686 lucene-spatial-4.3.0.jar 06/19/2013 02:47 AM 142,262 lucene-suggest-4.3.0.jar 04/23/2013 09:26 PM 211,185 Metadata-extractor-2.6.2.jar 06/19/2013 02:47 AM 85,309 metrics-core-3.0.0.jar 02/22/2013 08:08 AM 644,934 mina-core-2.0.4.jar 04/23/2013 09:25 PM 4,326,608 netcdf-4.2-min.jar 02/22/2013 08:08 AM 957,162 netty-3.4.0.Final.jar 06/19/2013 02:47 AM 22,545 noggit-0.5.jar 04/23/2013 09:25 PM 726,086 org.restlet-2.1.1.jar 04/23/2013 09:25 PM 19,672 org.restlet.ext.servlet-2.1.1.jar 02/22/2013 08:08 AM 29,555 paranamer-2.3.jar 04/23/2013 09:25 PM 3,910,899 pdfBox-1.7.1.jar 04/23/2013 09:26 PM 1,820,323 poi-3.8.jar 04/23/2013 09:26 PM 933,010 poi-ooxml-3.8.jar 04/23/2013 09:26 PM 4,706,775 poi-ooxml-schemas-3.8.jar 04/23/2013 09:26 PM 1,186,887 poi-scratchpad-3.8.jar 02/22/2013 08:08 AM 450,279 protobuf-java-2.4.1.jar 04/23/2013 09:26 PM 208,025 rome-0.9.jar 02/22/2013 08:08 AM 133,240 servlet-api-2.5-20110124.jar 02/22/2013 08:08 AM 25,496 slf4j-api-1.6.1.jar 02/22/2013 08:08 AM 9,753 slf4j-log4j12-1.6.1.jar 02/22/2013 08:08 AM 995,968 snappy-java-1.0.4.1.jar 06/23/2013 02:02 AM 29,880 solr-cell-4.3.0.jar 06/19/2013 02:47 AM 2,187,815 solr-core-4.3.0.jar 06/19/2013 02:47 AM 372,250 solr-solrj-4.3.0.jar 04/23/2013 09:25 PM 69,290 spatial4j-0.3.jar 04/23/2013 09:26 PM 90,722 tagsoup-1.2.1.jar 04/23/2013 09:25 PM 475,853 tika-core-1.3.jar 04/23/2013 09:25 PM 486,107 tika-parsers-1.3.jar 04/23/2013 09:25 PM 30,679 tika-xmp-1.3.jar 02/22/2013 08:08 AM 449,505 veLocity-1.7.jar 04/23/2013 09:25 PM 30,301 vorbis-java-core-0.1-tests.jar 04/23/2013 09:26 PM 47,478 vorbis-java-core-0.1.jar 04/23/2013 09:25 PM 14,752 vorbis-java-tika-0.1.jar 04/23/2013 09:25 PM 520,969 wstx-asl-3.2.7.jar 04/23/2013 09:26 PM 2,666,695 xmlbeans-2.3.0.jar 04/23/2013 09:26 PM 117,333 xmpcore-5.1.2.jar 02/22/2013 08:09 AM 94,672 xz-1.0.jar 05/08/2013 11:16 PM 608,239 zookeeper-3.3.6.jar 125 File(s) 65,826,273 bytes 3 Dir(s) 16,852,574,208 bytes free
C：用户克里斯 – >