General

Placeholder for KB's articles

org.postgresql.util.PSQLException: The connection attempt failed while running ingestion job in DataFoundry on Databricks
Problem Description: Postgres ingestion in DataFoundry on Databricks(AWS) fails with the below error in the databricks job log. 20/08/12 22:11:20 ER...
Thu, 13 Aug, 2020 at 3:43 AM
Crawl metadata failing with "Response Row size exceeds 64K bytes and is incompatible with the Client software" error in a Teradata source.
Problem Description:   Metacrawl of a Teradata source failing with below error if the row size exceeds 64K bytes, [ERROR] 2020-07-28 10:40:00,512 [mai...
Thu, 12 Nov, 2020 at 3:41 PM
org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block while running Spark Pipeline
Problem Description: Pipeline in cluster mode with Spark as execution engine will fail with the below error in the job log. Caused by: org.apache...
Thu, 30 Jul, 2020 at 7:07 AM
java.lang.NoClassDefFoundError: commons/exception/IWException wile doing Parquet source Metacrawl
Problem Description Metadata Crawl for a Parquet Source fails with the below error in the job log. Error: A JNI error has occurred, please check you...
Fri, 24 Jul, 2020 at 8:10 AM
How to register custom UDF in Infoworks?
Upload the jar which has custom logic to a folder in the Infoworks edge node. Jars related to Hadoop, spark, etc, will already be available on the Infoworks...
Wed, 22 Jul, 2020 at 8:17 AM
How to ingest data from Twitter using Infoworks?
These are the configurations required to ingest Twitter feed directly into your data lake using Infoworks. Step 1: Create a 'REST Generic' API ...
Mon, 3 Aug, 2020 at 10:22 PM
Upgrade to 3.1 fails with PermissionError: [Errno 13] Permission denied: ''
Problem Description: When upgrading infoworks to a newer version upgrade.sh script can fail with the following error trace 2020-06-15 16:15:05,...
Mon, 20 Jul, 2020 at 12:13 AM
Teradata ingestion fails with ERROR : Cannot resolve SQL type
Problem Description Teradata Ingestion fails with errors like below: org.apache.sqoop.orm.ClassWriter:581 :: Cannot resolve SQL type 2002 org.apach...
Mon, 24 Aug, 2020 at 9:14 AM
Incremental ingestion fails with "cannot perform MERGE as multiple source rows matched and attempted to update the same target row in the Delta table" error at merge phase.
Problem Description Incremental ingestion fails at the merge phase if the CDC data has multiple records for the given natural key with the below error. ...
Fri, 17 Jul, 2020 at 12:17 AM
Metadata crawl job fails for Teradata views with "Object does not exist" error
Problem Description: Metadata Crawl job fails for Teradata views with the below ERROR in the job log in Infoworks DataFoundry on Databricks. ERRO...
Mon, 3 Aug, 2020 at 10:24 PM