| SystemStructure
| Hive 에러 모음
에러1
[hadoop@hadoop conf]$ schematool -initSchema -dbType mysql
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
원인
위와 같은 에러는 HADOOP의 guaiva-*.jar와 HIVE guava-*.jar가 충돌나서 생기는 에러입니다.
해결
[HADOOP]
[hadoop@hadoop lib]$ cd $HADOOP_HOME/share/hadoop/hdfs/lib/
[hadoop@hadoop lib]$ pwd
/home/hadoop/hadoop/share/hadoop/hdfs/lib
[hadoop@hadoop lib]$ ls guava*
guava-27.0-jre.jar
[HIVE]
[hadoop@hadoop lib]$ cd $HIVE_HOME/lib/
[hadoop@hadoop lib]$ pwd
/home/hadoop/hive/lib
[hadoop@hadoop lib]$ ls guava*
guava-19.0.jar
버전이 낮은 HIVE의 guava*.jar파일을 삭제시키고, HADOOP의 guava*.jar를 복사하겠습니다.
[hadoop@hadoop lib]$ pwd
/home/hadoop/hive/lib
[hadoop@hadoop lib]$ rm guava-19.0.jar
[hadoop@hadoop lib]$ cp $HADOOP_HOME/share/hadoop/hdfs/lib/guava-27.0-jre.jar .
[hadoop@hadoop lib]$ ls guava*
guava-27.0-jre.jar
에러2
[hadoop@hadoop lib]$ schematool -initSchema -dbType mysql
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://namenode:3306/hivedb?createDatabaseIfNotExist=true
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: hiveuser
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: com.mysql.cj.jdbc.exceptions.CommunicationsException : Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
SQL Error code: 0
Use --verbose for detailed stacktrace.
*** schemaTool failed ***
원인 및 해결
알맞은 DB, 주소, Port번호 그리고 DB명이 입력되어있는지 확인한다.
cd $HIVE_HOME/conf/hive-site.xml
ls hive-site.xmljdbc:mysql://master:3306/hivedb?
vi hive-site.xml
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://master:3306/hivedb?createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>저는 master부분이 잘 못 되어있었음
에러3
[hadoop@hadoop conf]$ schematool -initSchema -dbType mysql
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://master:3306/hivedb?createDatabaseIfNotExist=true
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: hiveuser
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: java.sql.SQLException : null, message from server: "Host 'hadoop.master' is not allowed to connect to this MariaDB server"
SQL Error code: 1130
Use --verbose for detailed stacktrace.
*** schemaTool failed ***
원인
hiveuser에 권한이 없어서 생기는 에러이다.
해결
[hadoop@hadoop conf]$ mysql -uroot -pmysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 19
Server version: 10.5.16-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> grant all privileges on hivedb.* to hiveuser@'%' identified by 'hivepw';
Query OK, 0 rows affected (0.002 sec)
MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.000 sec)
MariaDB [(none)]> exit
Bye
'Cloud | AI | BigData' 카테고리의 다른 글
| [AWS-Hadoop|Hive|Spark] jupyter 설치 후 pyspark와 연동 (0) | 2022.12.14 |
|---|---|
| [AWS-Hadoop|Hive|Spark] Spark 3.2.3 설치 (0) | 2022.12.13 |
| [AWS-Hadoop|Hive|Spark] Hive 3.1.2 설치 (Mysql, Hadoop 연동) (0) | 2022.12.12 |
| [AWS-EC2 Hadoop|Hive|Spark] yum으로 MariaDB 10.5 설치 (0) | 2022.12.11 |
| [AWS-Hadoop|Hive|Spark] Hadoop 구축 핵심 및 에러 정리 (0) | 2022.12.10 |
댓글