Problem Description

Metadata Crawl for a Parquet Source fails with the below error in the job log.

Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: io/infoworks/saas/ingestion/commons/exception/IWException
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(
    at java.lang.Class.privateGetMethodRecursive(
    at java.lang.Class.getMethod0(
    at java.lang.Class.getMethod(
    at sun.launcher.LauncherHelper.validateMainClass(
    at sun.launcher.LauncherHelper.checkAndLoadMain(

Root Cause 

This issue happens if /opt/infoworks/lib/ingestion/commons.jar is missing in the ingestion_databricks_connect_classpath in file.


a) Append /opt/infoworks/lib/ingestion/commons.jar to the beginning of ingestion_databricks_connect_classpath  in $IW_HOME/conf/ file. This is under the assumption that /opt/infoworks is the home directory where Infoworks Data Foundry is installed.

b) cd $IW_HOME/bin
c) ./ ingestion-service
d) source
e) ./ ingestion-service
f) Run the Parquet ingestion. This should resolve the issue.

Applicable Infoworks Data Foundry on Databricks Versions: