问题描述
我正在尝试将文件从本地复制到 HDP（Hortonworks 数据平台）集群中的 hadoop。但我在 org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2116) 处收到连接拒绝异常 我是否需要更改集群中 hadoop 的任何配置？
示例程序：
String hdfsUrl = "hdfs://ambari-agent-1:8020";
Configuration configuration = new Configuration();
configuration.set("dfs.client.use.datanode.hostname","true");
FileSystem fs = FileSystem.get(new URI(hdfsUrl),configuration);
Path srcPath = new Path("C:\\HadoopPlugin\\SampleData\\salesrecord.csv");
Path destPath = new Path("hdfs://ambari-agent-1:8020/user/hadoop/testResult/");
fs.copyFromLocalFile(srcPath,destPath);
System.out.println("copied file successfully");
异常堆栈跟踪：
Exception in thread "main" java.io.IOException: Failed on local exception: java.io.IOException: An existing connection was forcibly closed by the remote host; Host Details : local host is: "DT01070442/192.168.44.7"; destination host is: "ambari-agent-1":8020;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:776)
at org.apache.hadoop.ipc.Client.call(Client.java:1480)
at org.apache.hadoop.ipc.Client.call(Client.java:1407)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy9.getFileInfo(UnkNown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy10.getFileInfo(UnkNown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2116)
at org.apache.hadoop.hdfs.distributedFileSystem$22.doCall(distributedFileSystem.java:1305)
at org.apache.hadoop.hdfs.distributedFileSystem$22.doCall(distributedFileSystem.java:1301)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.distributedFileSystem.getFileStatus(distributedFileSystem.java:1317)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424)
at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:496)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:348)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:338)
at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1965)
at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1933)
at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1898)
at practiceHadoop.copyFile.main(copyFile.java:19)
Caused by: java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.socketdispatcher.read0(Native Method)
at sun.nio.ch.socketdispatcher.read(Socketdispatcher.java:43)
at sun.nio.ch.IoUtil.readIntoNativeBuffer(IoUtil.java:223)
at sun.nio.ch.IoUtil.read(IoUtil.java:197)
at sun.nio.ch.socketChannelImpl.read(SocketChannelImpl.java:380)
at org.apache.hadoop.net.socketInputStream$Reader.performIO(SocketInputStream.java:57)
at org.apache.hadoop.net.socketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
at org.apache.hadoop.net.socketInputStream.read(SocketInputStream.java:161)
at org.apache.hadoop.net.socketInputStream.read(SocketInputStream.java:131)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:515)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
at java.io.DataInputStream.readInt(DataInputStream.java:387)
at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1079)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:974)
解决方法
这通常意味着远程端关闭了连接（通常是通过发送一个 TCP/IP RST 数据包）
可能的原因是： 网络问题 - 路由器、防火墙、线路 服务器（HOST）问题 - 资源不足，操作系统问题 服务问题-name node、jurnalnode、datanode服务
为了找到问题的原因，你必须先检查每一步的日志。
首先查看Hadoop服务日志
Hadoop 名称节点日志可以在 /var/hadoop/hadoop-hdfs-namenode-xxx.log 中找到 （位置和名称可能不同）
祝你好运！