General

Placeholder for KB's articles

How to ingest data from Twitter using Infoworks?
These are the configurations required to ingest Twitter feed directly into your data lake using Infoworks. Step 1: Create a 'REST Generic' API ...
Tue, 4 Aug, 2020 at 10:52 AM
Upgrade to 3.1 fails with PermissionError: [Errno 13] Permission denied: ''
Problem Description: When upgrading infoworks to a newer version upgrade.sh script can fail with the following error trace 2020-06-15 16:15:05,...
Mon, 20 Jul, 2020 at 12:43 PM
Teradata ingestion fails with ERROR : Cannot resolve SQL type
Problem Description Teradata Ingestion fails with errors like below: org.apache.sqoop.orm.ClassWriter:581 :: Cannot resolve SQL type 2002 org.apach...
Mon, 24 Aug, 2020 at 9:44 PM
Incremental ingestion fails with "cannot perform MERGE as multiple source rows matched and attempted to update the same target row in the Delta table" error at merge phase.
Problem Description Incremental ingestion fails at the merge phase if the CDC data has multiple records for the given natural key with the below error. ...
Fri, 17 Jul, 2020 at 12:47 PM
Metadata crawl job fails for Teradata views with "Object does not exist" error
Problem Description: Metadata Crawl job fails for Teradata views with the below ERROR in the job log in Infoworks DataFoundry on Databricks. ERRO...
Tue, 4 Aug, 2020 at 10:54 AM
Sample Post Ingestion Hook Script
Description Post hook ingestion is a feature where you can provide bash scripts to be run after the ingestion job completes. The post hook runs in the s...
Mon, 13 Jul, 2020 at 12:02 PM
Script for deleting jobs in Databricks workspace for 1000 job limit/workspace
Problem Description Whenever we submit the Ingestion jobs, the Pipeline jobs or the Export Jobs from Infoworks, Infoworks runs these jobs on the Databri...
Fri, 9 Oct, 2020 at 9:02 PM
XML based rest API ingestion fails with java.lang.NoClassDefFoundError: org/json/XML
Problem Description XML based rest API ingestion fails with below error Caused by: java.util.concurrent.ExecutionException: Boxed Error at scala.co...
Fri, 10 Jul, 2020 at 1:14 PM
Unexpected character (') while crawling schema in REST API source
Problem Description: Schema crawl in REST API Generic source fails with the below error in $IW_HOME/platform/logs/queryservice.log 020-07-08 09:31:3...
Wed, 8 Jul, 2020 at 3:12 PM
Spark pipeline build job fails with SASL AUTHENTICATION ERROR
Problem Description: Spark pipeline build job fails with then below SASL Authentication error in the job log in Infoworks DataFoundry EMR 3.1.1. Fai...
Wed, 8 Jul, 2020 at 2:57 PM