General

Placeholder for KB's articles

How to create Generic JDBC Excel source in Infoworks DataFoundry 4.2
Description: Infoworks DataFoundry supports extracting data from an Excel file using Generic JDBC Source. a) Go to Admin>Sources>Add New Source an...
Wed, 16 Sep, 2020 at 11:31 AM
How to add custom tags to Infoworks Job clusters?
Description By default the Infoworks Job clusters spun up for ingestion or pipelines jobs will have a few default tags If customers wish to add a few c...
Thu, 4 Aug, 2022 at 3:47 AM
Ingest job fails at merge stage with Table or view not found error
Problem statement: Ingest job fails at merge state with following error message in the databricks log  20/08/18 18:46:43 ERROR DistJobsDriver: org.apach...
Fri, 1 Oct, 2021 at 11:41 AM
Kafka ingestion fails with error Caused by: java.lang.NoSuchMethodError: com.univocity.parsers.csv.CsvFormat.setDelimiter
Problem Description: Kafka streaming ingestion fails with below error Caused by: java.lang.NoSuchMethodError: com.univocity.parsers.csv.CsvFormat.se...
Mon, 24 Aug, 2020 at 9:42 PM
Steps to take backup of MongoDb data and restore the data from the backup file
Description: Below are the steps to take the backup of Infoworks DataFoundry metadata which is stored in MongoDB and to restore the same.  Note: Ensure...
Tue, 12 Jul, 2022 at 2:52 AM
Crawl metadata failing with "Response Row size exceeds 64K bytes and is incompatible with the Client software" error in a Teradata source.
Problem Description:   Metacrawl of a Teradata source failing with below error if the row size exceeds 64K bytes, [ERROR] 2020-07-28 10:40:00,512 [mai...
Sat, 4 Sep, 2021 at 1:24 AM
java.lang.NoClassDefFoundError: commons/exception/IWException wile doing Parquet source Metacrawl
Problem Description Metadata Crawl for a Parquet Source fails with the below error in the job log. Error: A JNI error has occurred, please check you...
Fri, 24 Jul, 2020 at 8:40 PM
Incremental ingestion fails with "cannot perform MERGE as multiple source rows matched and attempted to update the same target row in the Delta table" error at merge phase.
Problem Description Incremental ingestion fails at the merge phase if the CDC data has multiple records for the given natural key with the below error. ...
Fri, 17 Jul, 2020 at 12:47 PM
Sample Post Ingestion Hook Script
Description Post hook ingestion is a feature where you can provide bash scripts to be run after the ingestion job completes. The post hook runs in the s...
Mon, 13 Jul, 2020 at 12:02 PM
Script for deleting jobs in Databricks workspace for 1000 job limit/workspace
Problem Description Whenever we submit the Ingestion jobs, the Pipeline jobs or the Export Jobs from Infoworks, Infoworks runs these jobs on the Databri...
Fri, 9 Oct, 2020 at 9:02 PM