Problem Description: Ingestion jobs fail while launching a yarn application. Application log fails with the following error message

2020-08-05 22:22:18,982 WARN [main] org.mortbay.log: failed guice: java.lang.IncompatibleClassChangeError: Implementing class

2020-08-05 22:22:18,982 WARN [main] org.mortbay.log: failed
org.mortbay.jetty.webapp.WebAppContext@718dbd79{/,jar:file:/hadoop/yarn/local/filecache/23/mapreduce.tar.gz/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.7.3.2.6.4.0-91.jar!/webapps/mapreduce}: java.lang.IncompatibleClassChangeError: Implementing class

2020-08-05 22:22:18,983 WARN [main] org.mortbay.log: failed ContextHandlerCollection@706fe5c6: java.lang.IncompatibleClassChangeError: Implementing class

2020-08-05 22:22:18,984 ERROR [main] org.mortbay.log: Error starting handlersjava.lang.IncompatibleClassChangeError: Implementing class        
at java.lang.ClassLoader.defineClass1(Native Method)        
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)



Root cause: The error is due to the conflicting jetty jars that are getting loaded. The error is occurring while implementing the class org.mortbay.jetty.webapp.WebAppContext

Solution: 

Among the jars that are getting loaded following jars have the above class

1. jetty-all-9.4.18.v20190429-uber.jar
2. jetty-all-7.6.0.v20120127.jar
3. jetty-all-server-7.6.0.v20120127.jar

The last two are the native jars that come with hdp while the first one is shipped along with Infoworks DF and it is present in $IW_HOME/bin/shared 

Steps: 

1. Move the $IW_HOME/bin/shared/jetty-all-9.4.18.v20190429-uber.jar to a temporary location out $IW_HOME
2. cat /opt/infoworks/conf/conf.properties | grep iw_hdfs_temp_home (This will return you the temp location on hdfs)

3. hadoop fs -rm -r <location from previous step>/temp/iwjobs/libjars/
4. Trigger the ingestion jobs


Applicable Infoworks DataFoundry Versions:

v.2.9