Problem Description:


Metadata Crawl job fails for Teradata views with the below ERROR in the job log in Infoworks DataFoundry on Databricks.



ERROR] 2020-07-08 09:17:04,776 [main] io.infoworks.saas.ingestion.core.metacrawl.sources.RDBMSMetaCrawler:396 :: Error while fetching metadata for CUST_DEX_PARM

io.infoworks.saas.ingestion.commons.exception.IWException: java.sql.SQLException: [Teradata Database] [TeraJDBC 15.00.00.20] [Error 3807] [SQLState 42S02] Object 'DEX_PARM' does not exist.


ERROR] 2020-07-08 09:16:12,568 [main] io.infoworks.saas.ingestion.core.metacrawl.sources.RDBMSMetaCrawler:396 :: Error while fetching metadata for BI_LATAM_SLS_IN_AGG

io.infoworks.saas.ingestion.commons.exception.IWException: java.sql.SQLException: [Teradata Database] [TeraJDBC 15.00.00.20] [Error 3810] [SQLState 42S22] Column/Parameter 'DWL_P_INTL_WORK.BI_LATAM_SLS_IN_AGG.SLS_WK' does not exist.


Root cause:


This issue happens if the underlying Teradata database tables/columns on which these views are created are dropped from the database end.


You could run a select query on the Teradata database to verify the same. Run a simple select * from on the tables that are mentioned in the log messages.



Solution/Workaround:


This is a limitation in Infoworks DataFoundry on Databricks v3.1,v3.2. Contact Infoworks Support to get the workaround for this issue.


Applicable Infoworks DataFoundry on Databricks versions:


v3.1,v3.2