General

Placeholder for KB's articles

Teradata ingestion fails with ERROR : Cannot resolve SQL type
Problem Description Teradata Ingestion fails with errors like below: org.apache.sqoop.orm.ClassWriter:581 :: Cannot resolve SQL type 2002 org.apach...
Mon, 24 Aug, 2020 at 9:14 AM
Incremental ingestion fails with "cannot perform MERGE as multiple source rows matched and attempted to update the same target row in the Delta table" error at merge phase.
Problem Description Incremental ingestion fails at the merge phase if the CDC data has multiple records for the given natural key with the below error. ...
Fri, 17 Jul, 2020 at 12:17 AM
Metadata crawl job fails for Teradata views with "Object does not exist" error
Problem Description: Metadata Crawl job fails for Teradata views with the below ERROR in the job log in Infoworks DataFoundry on Databricks. ERRO...
Mon, 3 Aug, 2020 at 10:24 PM
Sample Post Ingestion Hook Script
Description Post hook ingestion is a feature where you can provide bash scripts to be run after the ingestion job completes. The post hook runs in the s...
Sun, 12 Jul, 2020 at 11:32 PM
Script for deleting jobs in Databricks workspace for 1000 job limit/workspace
Problem Description Whenever we submit the Ingestion jobs, the Pipeline jobs or the Export Jobs from Infoworks, Infoworks runs these jobs on the Databri...
Fri, 9 Oct, 2020 at 8:32 AM
XML based rest API ingestion fails with java.lang.NoClassDefFoundError: org/json/XML
Problem Description XML based rest API ingestion fails with below error Caused by: java.util.concurrent.ExecutionException: Boxed Error at scala.co...
Fri, 10 Jul, 2020 at 12:44 AM
Unexpected character (') while crawling schema in REST API source
Problem Description: Schema crawl in REST API Generic source fails with the below error in $IW_HOME/platform/logs/queryservice.log 020-07-08 09:31:3...
Wed, 8 Jul, 2020 at 2:42 AM
Spark pipeline build job fails with SASL AUTHENTICATION ERROR
Problem Description: Spark pipeline build job fails with then below SASL Authentication error in the job log in Infoworks DataFoundry EMR 3.1.1. Fai...
Wed, 8 Jul, 2020 at 2:27 AM
Spark Pipeline build job process 0 records in Infoworks DataFoundry 3.1.1-EMR
Problem Description Pipeline build job will be successful but it will process 0 records in Infoworks DataFoundry 3.1.1-EMR Root cause: This iss...
Wed, 8 Jul, 2020 at 2:17 AM
Mongo HA Setup - Configure optimal oplog sizes
Problem Description  Infoworks mongo metadata database runs in a three node master slave setup when HA is enabled. If service on one node is down for an...
Wed, 8 Jul, 2020 at 9:27 PM