The specific errors encountered in building Hadoop HA are as follows
com.jcraft.jsch.JSchException: Auth fail
at com.jcraft.jsch.Session.connect(Session.java:452)
at org.apache.hadoop.ha.SshFenceByTcpPort.tryFence(SshFenceByTcpPort.java:100)
at org.apache.hadoop.ha.NodeFencer.fence(NodeFencer.java:97)
at org.apache.hadoop.ha.ZKFailoverController.doFence(ZKFailoverController.java:532)
at org.apache.hadoop.ha.ZKFailoverController.fenceOldActive(ZKFailoverController.java:505)
at org.apache.hadoop.ha.ZKFailoverController.access$1100(ZKFailoverController.java:61)
at org.apache.hadoop.ha.ZKFailoverController$ElectorCallbacks.fenceOldActive(ZKFailoverController.java:892)
at org.apache.hadoop.ha.ActiveStandbyElector.fenceOldActive(ActiveStandbyElector.java:902)
at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:801)
at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:416)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:599)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
2021-12-27 11:07:20,846 WARN org.apache.hadoop.ha.NodeFencer: Fencing method org.apache.hadoop.ha.SshFenceByTcpPort(null) was unsuccessful.
2021-12-27 11:07:20,846 ERROR org.apache.hadoop.ha.NodeFencer: Unable to fence service by any configured method.
2021-12-27 11:07:20,846 WARN org.apache.hadoop.ha.ActiveStandbyElector: Exception handling the winning of election
java.lang.RuntimeException: Unable to fence NameNode at a/192.168.0.149:8020
at org.apache.hadoop.ha.ZKFailoverController.doFence(ZKFailoverController.java:533)
at org.apache.hadoop.ha.ZKFailoverController.fenceOldActive(ZKFailoverController.java:505)
at org.apache.hadoop.ha.ZKFailoverController.access$1100(ZKFailoverController.java:61)
at org.apache.hadoop.ha.ZKFailoverController$ElectorCallbacks.fenceOldActive(ZKFailoverController.java:892)
at org.apache.hadoop.ha.ActiveStandbyElector.fenceOldActive(ActiveStandbyElector.java:902)
at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:801)
at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:416)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:599)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
2021-12-27 11:07:20,846 INFO org.apache.hadoop.ha.ActiveStandbyElector: Trying to re-establish ZK session
2021-12-27 11:07:20,851 INFO org.apache.zookeeper.ZooKeeper: Session: 0x37df9b417310059 closed
2021-12-27 11:07:21,852 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=a:2181,b:2181,c:2181 sessionTimeout=5000 watcher[email protected]44a90199
2021-12-27 11:07:21,853 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server b/192.168.0.150:2181. Will not attempt to authenticate using SASL (unknown error)
2021-12-27 11:07:21,854 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to b/192.168.0.150:2181, initiating session
2021-12-27 11:07:21,859 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server b/192.168.0.150:2181, sessionid = 0x27df9b3aaf60068, negotiated timeout = 5000
2021-12-27 11:07:21,860 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2021-12-27 11:07:21,861 INFO org.apache.hadoop.ha.ActiveStandbyElector: Session connected.
2021-12-27 11:07:21,862 INFO org.apache.hadoop.ha.ActiveStandbyElector: Checking for any old active which needs to be fenced...
2021-12-27 11:07:21,862 INFO org.apache.hadoop.ha.ActiveStandbyElector: Old node exists: 0a096d79636c757374657212026e311a016120d43e28d33e
2021-12-27 11:07:21,864 INFO org.apache.hadoop.ha.ZKFailoverController: Should fence: NameNode at a/192.168.0.149:8020
2021-12-27 11:07:22,866 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: a/192.168.0.149:8020. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=1, sleepTime=1000 MILLISECONDS)
2021-12-27 11:07:22,867 WARN org.apache.hadoop.ha.FailoverController: Unable to gracefully make NameNode at a/192.168.0.149:8020 standby (unable to connect)
java.net.ConnectException: Call From b/192.168.0.150 to a:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.GeneratedConstructorAccessor26.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
at org.apache.hadoop.ipc.Client.call(Client.java:1480)
at org.apache.hadoop.ipc.Client.call(Client.java:1407)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy9.transitionToStandby(Unknown Source)
at org.apache.hadoop.ha.protocolPB.HAServiceProtocolClientSideTranslatorPB.transitionToStandby(HAServiceProtocolClientSideTranslatorPB.java:112)
at org.apache.hadoop.ha.FailoverController.tryGracefulFence(FailoverController.java:172)
at org.apache.hadoop.ha.ZKFailoverController.doFence(ZKFailoverController.java:514)
at org.apache.hadoop.ha.ZKFailoverController.fenceOldActive(ZKFailoverController.java:505)
at org.apache.hadoop.ha.ZKFailoverController.access$1100(ZKFailoverController.java:61)
at org.apache.hadoop.ha.ZKFailoverController$ElectorCallbacks.fenceOldActive(ZKFailoverController.java:892)
at org.apache.hadoop.ha.ActiveStandbyElector.fenceOldActive(ActiveStandbyElector.java:902)
at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:801)
at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:416)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:599)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:609)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:707)
at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:370)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1529)
at org.apache.hadoop.ipc.Client.call(Client.java:1446)
... 14 more
Here are two possible reasons for this error. You are also welcome to point out the shortcomings and discuss with us.
The first is that SSH secret login is not configured. You can try to report an error and log in with other machines to see if you can successfully log in without secret.
The second is because the parameter of dfs.ha.fencing.methods
is sshence, and needs to use fuser command; maybe you do not install fuser
(required for each namenode node)
installation command: Yum - y install psmisc
Read More:
- [Solved] Zookeeper cluster error: Error contacting service It is probably not running. (alicloud server)
- [Solved] Sqoop Error: ERROR tool.ImportTool: Import failed: java.io.IOException
- [Solved] JAVA connect HBase program is stuck and does not report an error
- [Solved] hadoop Error: 9000 failed on connection exception java.net.ConnectException Denied to Access
- ZK Error contacting service. It is probably not running [How to Solve]
- [Solved] java.io.IOException: Got error, status=ERROR, status message, ack with firstBadLink as
- Spring integrated HBase error [How to Solve]
- [Solved] Mapreducer Class Conversion error: java.lang.ClassCastException
- Run hadoop fs -put Command Error: java.io.IOException: Got error, status message , ack with firstBadLink
- [Solved] Hive On Spark Error: Remote Spark Driver – HiveServer2 connection has been closed
- [Solved] Rocketmq remote connection error: sendDefaultimpl call timeout
- [Solved] Hadoop Error: HADOOP_HOME and hadoop.home.dir are unset.
- [Solved] Solr8 establishes the cluster node as active, but the query reports an error
- Hadoop Startup Error: sbin/start-dfs.sh [How to Solve]
- [Solved] org.apache.flink.client.program.ProgramInvocationException: The main method caused an error
- [Solved] Hbase Error: ERROR: KeeperErrorCode = NoNode for /hbase/master
- [Solved] Docker-maven-plugin Build Mirror Error: failed: Connection refused: connect
- How to Solve classnotfoundexception error in spark without Hadoop runtime
- The java springboot websocket service server actively closes the connection and causes java.io.EOFException to be thrown
- Build a mybatis and it will appear session.selectOne Method error