PROBLEM DESCRIPTION


Please refer this article if the ingestion job fails with the below error in the logs

java.lang.NoClassDefFoundError: org/apache/hive/service/rpc/thrift/TCLIService$Iface
            at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
            at java.sql.DriverManager.getConnection(DriverManager.java:664)
            at java.sql.DriverManager.getConnection(DriverManager.java:247)
            at infoworks.tools.hive.HiveUtils$1.run(HiveUtils.java:602)
            at infoworks.tools.hive.HiveUtils$1.run(HiveUtils.java:599)
            at infoworks.tools.security.authentication.AuthenticationUtil.doAs(AuthenticationUtil.java:223)
            at infoworks.tools.security.authentication.AuthenticationUtil.doAs(AuthenticationUtil.java:204)
            at infoworks.tools.hive.HiveUtils.getConnection(HiveUtils.java:599)
            at infoworks.tools.hive.HiveUtils.getConnection(HiveUtils.java:563)
            at infoworks.tools.hive.HiveUtils.execStmts(HiveUtils.java:453)
            at infoworks.tools.hive.HiveUtils.dropEntityIfExists(HiveUtils.java:1172)
            at infoworks.tools.hive.HiveUtils.dropTableIfExists(HiveUtils.java:1181)
             at 
infoworks.discovery.dbcrawler.rdbms.CrawlController.cleanFoldersAndHiveForTable(CrawlController.java:1284


CAUSE


This issue occurs if the hive-service-rpc-2.1.1.jar is not added to iw_jobs_classpath in conf.properties file.


WORKAROUND/ RESOLUTION


  1. Go to $IW_HOME/conf directory and open the conf.propeties file.
  2. Append the absolute path  hive-service-rpc-2.1.1.jar for to the iw_jobs_classpath as shown below and then run the ingestion job. 
    iw_jobs_classpath=/opt/infoworks/lib/schemacrawler-12.06.03-main/schemacrawler-12.06.03.jar:/opt/infoworks/lib/mongodb/mongo-java-driver-2.12.3.jar:/opt/infoworks/bin/tools.jar:/opt/infoworks/lib/exec/commons-exec-1.2.jar:/opt/infoworks/lib/teradata/tdgssconfig.jar:/opt/infoworks/lib/teradata/teradata-connector-1.1.1.jar:/opt/infoworks/lib/teradata/terajdbc4.jar:/opt/infoworks/lib/mongodblogger/logger.jar:/opt/infoworks/lib/mongodblogger/log4mongo-java-0.7.4.jar:/opt/infoworks/lib/antlr/*:/opt/infoworks/lib/jackson/*:/opt/infoworks/lib/jsqlparser/*:/opt/infoworks/lib/shared/*:/opt/infoworks/lib/jwt/*:/usr/local/hive/conf/:/usr/local/hive/lib/*:/usr/local/hive/lib/hive-serde-2.1.1.jar:/usr/local/hive/lib/hive-metastore-2.1.1.jar:/usr/local/hive/lib/hive-common-2.1.1.jar:/usr/local/hive/lib/libthrift-0.9.3.jar:/usr/local/hive/lib/httpclient-4.4.jar:/usr/local/hive/lib/httpcore-4.4.jar:/usr/local/hive/lib/hive-cli-2.1.1.jar:/usr/local/hive/lib/hive-service-2.1.1.jar:/usr/local/hive/lib/hive-service-rpc-2.1.1.jar:/usr/local/hive/lib/hive-jdbc-2.1.1.jar:/usr/local/hive/hcatalog/share/hcatalog/*:/usr/lib/hive/conf:/opt/infoworks/lib/commons-lang3/*


Note: hive-service-rpc-2.1.1.jar is for Hive 2.1.1. Use the relevant jar for the hive version that you are using.


APPLIES TO VERSIONS : 2.3.x and 2.4.x