preface
This article belongs to the column “big data abnormal problems summary”, which is the author’s original. Please indicate the source of the quotation, and point out the shortcomings and errors in the comments area. Thank you!
For the table of contents and references of this column, please refer to the summary of big data anomalies
Questions
When installing and deploying hive3.1.2, an error occurred during startup:
[[email protected] apache-hive-3.1.2-bin]# ./bin/hive
which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/java/bin:/opt/maven/bin:/opt/bigdata/hadoop-3.2.2/bin:/opt/bigdata/hadoop-3.2.2/sbin:/opt/bigdata/spark-3.2.0/bin:/opt/bigdata/spark-3.2.0/sbin:/opt/java/bin:/opt/maven/bin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/bigdata/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/bigdata/hadoop-3.2.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:536)
at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:554)
at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:448)
at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5141)
at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:5099)
at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:97)
at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:81)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:699)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
solve
From the error log, we can see that there is a problem with the guava package. Either there are fewer packages or there are conflicts. Let’s first check whether the package exists
[[email protected] apache-hive-3.1.2-bin]# find/-name *guava*
find: ‘/proc/9666’: Not having that file or directory
/opt/maven/lib/guava.license
/opt/maven/lib/guava-25.1-android.jar
/opt/bigdata/hadoop-2.7.3/share/hadoop/common/lib/guava-11.0.2.jar
/opt/bigdata/hadoop-2.7.3/share/hadoop/hdfs/lib/guava-11.0.2.jar
/opt/bigdata/hadoop-2.7.3/share/hadoop/httpfs/tomcat/webapps/webhdfs/WEB-INF/lib/guava-11.0.2.jar
/opt/bigdata/hadoop-2.7.3/share/hadoop/kms/tomcat/webapps/kms/WEB-INF/lib/guava-11.0.2.jar
/opt/bigdata/hadoop-2.7.3/share/hadoop/yarn/lib/guava-11.0.2.jar
/opt/bigdata/hadoop-2.7.3/share/hadoop/tools/lib/guava-11.0.2.jar
/opt/bigdata/hadoop-3.2.2/share/hadoop/common/lib/guava-27.0-jre.jar
/opt/bigdata/hadoop-3.2.2/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
/opt/bigdata/hadoop-3.2.2/share/hadoop/hdfs/lib/guava-27.0-jre.jar
/opt/bigdata/hadoop-3.2.2/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
/opt/bigdata/spark-3.2.0/jars/guava-14.0.1.jar
/opt/bigdata/apache-hive-3.1.2-bin/lib/guava-19.0.jar
/opt/bigdata/apache-hive-3.1.2-bin/lib/jersey-guava-2.25.1.jar
There are guava packages in hive’s lib directory and Hadoop’s common lib directory, but the package versions are inconsistent. We use guava-27.0-jre.jar to override guava-19.0.jar in hive’s lib directory
[[email protected] apache-hive-3.1.2-bin]# cd lib
[[email protected] lib]# ll *guava*
-rwxr-xr-x. 1 hadoop hadoop 2308517 Sep 27 2018 guava-19.0.jar
-rwxr-xr-x. 1 hadoop hadoop 971309 May 21 2019 jersey-guava-2.25.1.jar
[[email protected] lib]# cd /opt/bigdata/hadoop-3.2.2/share/hadoop/common/lib/
[[email protected] lib]# ll *guava*
-rwxr-xr-x. 1 hadoop hadoop 2747878 Sep 26 00:18 guava-27.0-jre.jar
-rwxr-xr-x. 1 hadoop hadoop 2199 Sep 26 00:18 listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
[[email protected] lib]# cd /opt/bigdata/apache-hive-3.1.2-bin/lib/
[[email protected] lib]# mv guava-19.0.jar guava-19.0.jar.bak
[[email protected] lib]# cp /opt/bigdata/hadoop-3.2.2/share/hadoop/common/lib/guava-27.0-jre.jar /opt/bigdata/apache-hive-3.1.2-bin/lib/
[[email protected] lib]# ll *guava*
-rwxr-xr-x. 1 hadoop hadoop 2308517 Sep 27 2018 guava-19.0.jar.bak
-rwxr-xr-x. 1 root root 2747878 Jun 19 13:23 guava-27.0-jre.jar
-rwxr-xr-x. 1 hadoop hadoop 971309 May 21 2019 jersey-guava-2.25.1.jar
[[email protected] lib]# chmod 755 guava-27.0-jre.jar
[[email protected] lib]# chown hadoop:hadoop guava-27.0-jre.jar
At the same time, we switch to Hadoop users to execute (this can ensure that Hadoop is a super user)
[[email protected] lib]# su - hadoop
Last login: April June 17 22:14:01 CST 2021pts/0 AM
[[email protected] ~]$ cd /opt/bigdata/apache-hive-3.1.2-bin/
[[email protected] apache-hive-3.1.2-bin]$ ./bin/hive
which: no hbase in (/home/hadoop/.local/bin:/home/hadoop/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/java/bin:/opt/maven/bin:/opt/bigdata/hadoop-3.2.2/bin:/opt/bigdata/hadoop-3.2.2/sbin:/opt/bigdata/spark-3.2.0/bin:/opt/bigdata/spark-3.2.0/sbin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/bigdata/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/bigdata/hadoop-3.2.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 901d74e6-fa5d-45b5-b949-bebdacb582ed
Logging initialized using configuration in jar:file:/opt/bigdata/apache-hive-3.1.2-bin/lib/hive-common-3.1.2.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive>
The problem has been solved
Read More:
- [Solved] Hive Warehouse error: com.google.common.base.Preconditions.checkArgument
- Initial installation of hive-2.1.0 startup error problem solution
- [Solved] bin/hive Startup Error: Operation category READ is not supported in state standby
- [ERROR] Terminal initialization failed; falling back to unsupported java.lang.IncompatibleClassChang
- [Solved] Exception in thread “main“ org.apache.spark.SparkException: When running with master ‘yarn‘ either
- How to Solve Error in importing scala word2vecmodel
- Centos7 hive started to report an error. There is no route to the host. The firewall has been closed
- [Solved] Hive Error while processing statement: FAILED: Execution Error
- [Solved] Error: Could not open client transport with JDBC Uri
- [Solved] Hive tez due to: ROOT_INPUT_INIT_FAILURE java.lang.IllegalArgumentException: Illegal Capacity: -38297
- How to Solve HiveServer2 & Beeline Error
- Sqoop exports hive data to MySQL Error [How to Solve]
- [Solved] Error while processing statement: FAILED: Execution Error, return code 3 from org.apache.
- Hive ERROR Failed with exception java.io.IOException:java.lang.IllegalArgumentException
- [Solved] Schema initialization FAILED Metastore state would be inconsistent
- [Solved] Hadoop Error: Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
- How to Solve Spark Writes Hudi Error
- How to Solve HBase error: region is not online
- [Solved] Hadoop Mapreduce Error: GC overhead limit exceeded