[AWS-Hadoop|Hive|Spark] Hive 에러 모음

Cloud | AI | BigData

[AWS-Hadoop|Hive|Spark] Hive 에러 모음

개폰지밥 2022. 12. 12. 13:26

| SystemStructure

| Hive 에러 모음

에러1

[hadoop@hadoop conf]$ schematool -initSchema -dbType mysql

Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V

원인

위와 같은 에러는 HADOOP의 guaiva-*.jar와 HIVE guava-*.jar가 충돌나서 생기는 에러입니다.

해결

[HADOOP]

[hadoop@hadoop lib]$ cd $HADOOP_HOME/share/hadoop/hdfs/lib/
[hadoop@hadoop lib]$ pwd
/home/hadoop/hadoop/share/hadoop/hdfs/lib
[hadoop@hadoop lib]$ ls guava*
guava-27.0-jre.jar

[HIVE]

[hadoop@hadoop lib]$ cd $HIVE_HOME/lib/
[hadoop@hadoop lib]$ pwd
/home/hadoop/hive/lib
[hadoop@hadoop lib]$ ls guava*
guava-19.0.jar

버전이 낮은 HIVE의 guava*.jar파일을 삭제시키고, HADOOP의 guava*.jar를 복사하겠습니다.

[hadoop@hadoop lib]$ pwd
/home/hadoop/hive/lib

[hadoop@hadoop lib]$ rm guava-19.0.jar

[hadoop@hadoop lib]$ cp $HADOOP_HOME/share/hadoop/hdfs/lib/guava-27.0-jre.jar .

[hadoop@hadoop lib]$ ls guava*
guava-27.0-jre.jar

에러2

[hadoop@hadoop lib]$  schematool -initSchema -dbType mysql
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:mysql://namenode:3306/hivedb?createDatabaseIfNotExist=true
Metastore Connection Driver :    com.mysql.jdbc.Driver
Metastore connection User:       hiveuser
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: com.mysql.cj.jdbc.exceptions.CommunicationsException : Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
SQL Error code: 0
Use --verbose for detailed stacktrace.
*** schemaTool failed ***

원인 및 해결

알맞은 DB, 주소, Port번호 그리고 DB명이 입력되어있는지 확인한다.

cd $HIVE_HOME/conf/hive-site.xml

ls hive-site.xml

jdbc:mysql://master:3306/hivedb?

vi hive-site.xml

 <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://master:3306/hivedb?createDatabaseIfNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
  </property>

저는 master부분이 잘 못 되어있었음

에러3

[hadoop@hadoop conf]$ schematool -initSchema -dbType mysql
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:mysql://master:3306/hivedb?createDatabaseIfNotExist=true
Metastore Connection Driver :    com.mysql.jdbc.Driver
Metastore connection User:       hiveuser
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: java.sql.SQLException : null,  message from server: "Host 'hadoop.master' is not allowed to connect to this MariaDB server"
SQL Error code: 1130
Use --verbose for detailed stacktrace.
*** schemaTool failed ***

원인

hiveuser에 권한이 없어서 생기는 에러이다.

해결

[hadoop@hadoop conf]$ mysql -uroot -pmysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 19
Server version: 10.5.16-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> grant all privileges on hivedb.* to hiveuser@'%' identified by 'hivepw';
Query OK, 0 rows affected (0.002 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.000 sec)

MariaDB [(none)]> exit
Bye

저작자표시 비영리 변경금지