PROBLEM DESCRIPTION


While exporting a pipeline to any database, the job fails with the below error in the logs


INFO] 2018-03-30 09:49:27,883 [main] infoworks.tools.progress.ProgressUtil:182 :: Caught exception: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/MetaException

[ERROR] 2018-03-30 09:49:27,886 [main] infoworks.discovery.export.Driver:269 :: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/MetaException

at java.util.concurrent.FutureTask.report(FutureTask.java:122)

at java.util.concurrent.FutureTask.get(FutureTask.java:192)

at infoworks.discovery.export.Driver.main(Driver.java:255)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.hadoop.util.RunJar.run(RunJar.java:233)

at org.apache.hadoop.util.RunJar.main(RunJar.java:148)

Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/MetaException

at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:478)

at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:201)

at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:176)

at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)

at java.sql.DriverManager.getConnection(DriverManager.java:664)

at java.sql.DriverManager.getConnection(DriverManager.java:247)

at infoworks.tools.hive.HiveUtils$1.run(HiveUtils.java:598)

at infoworks.tools.hive.HiveUtils$1.run(HiveUtils.java:595)

at infoworks.tools.security.authentication.AuthenticationUtil.doAs(AuthenticationUtil.java:223)

at infoworks.tools.security.authentication.AuthenticationUtil.doAs(AuthenticationUtil.java:204)

at infoworks.tools.hive.HiveUtils.getConnection(HiveUtils.java:595)

at infoworks.tools.hive.HiveUtils.getConnection(HiveUtils.java:559)

at infoworks.tools.hive.HiveUtils.execStmts(HiveUtils.java:449)

at infoworks.tools.hive.HiveUtils.dropTable(HiveUtils.java:1156)

at infoworks.discovery.export.rdbms.destination.TeradataDestination.dropTempHiveTable(TeradataDestination.java:572)

at infoworks.discovery.export.rdbms.destination.TeradataDestination.export(TeradataDestination.java:654)

at infoworks.discovery.export.ExportWorkerThread.call(ExportWorkerThread.java:36)

at infoworks.discovery.export.ExportWorkerThread.call(ExportWorkerThread.java:7)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.metastore.api.MetaException

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

... 22 more


CAUSE


This issue occurs if the hive-metastore.jar is missing in the iw_export_classpath


RESOLUTION:


a) Go to $IW_HOME/conf directory

b) Open the conf.properties file and append the below jar to the iw_export_classpath variable


 /usr/hdp/current/hive-client/lib/hive-metastore.jar


c) Save the file and rerun the export job.