Problem Description:

 

Metacrawl of a Teradata source failing with below error if the row size exceeds 64K bytes,

[ERROR] 2020-07-28 10:40:00,512 [main] io.infoworks.saas.ingestion.core.metacrawl.sources.RDBMSMetaCrawler:396 :: Error while fetching metadata for demo_cl

io.infoworks.saas.ingestion.commons.exception.IWException: java.sql.SQLException: [Teradata Database] [TeraJDBC 15.00.00.20] [Error 9990] [SQLState HY000] Response Row size exceeds 64K bytes and is incompatible with the Client software.


Root cause:


Error 9990 points that response row size exceeds 64K bytes limit. This issue will occur if the Teradata drivers are in version 15.10.


Solution/Workaround for Infoworks Datafoundry v3.x


Upgrading the Teradata drives to version 16.10 will resolve this issue.

Steps: 

  • Login to the Infoworks edge node as Infoworks user and navigate to $IW_HOME/lib/ingestion/jdbc.
  • Move terajdbc4.jar and tdgssconfig.jar jars out of $IW_HOME/lib/ingestion/jdbc.
  • Add below attached jars to $IW_HOME/lib/ingestion/jdbc directory.
  • Rerun the metacrawl.


Applicable Infoworks DataFoundry on Databricks versions:


Solution/Workaround for Infoworks Datafoundry v5.x


  • Login to the Infoworks edge node as Infoworks user and navigate to /opt/infoworks/lib/ingestion/connectors/teradata/lib/dist-jobs/
  • Move terajdbc4.jar and tdgssconfig.jar jars out of /opt/infoworks/lib/ingestion/connectors/teradata/lib/dist-jobs/
  • Add below attached jars to /opt/infoworks/lib/ingestion/connectors/teradata/lib/dist-jobs/ directory.
  • Rerun the metacrawl.