General

Placeholder for KB's articles

Provisioning Dataproc Cluster's with custom machine image through Infoworks Creating a custom Dataproc Image Google provides an option to launch Dataproc cluster with custom machine image Custom machine image will decrease the cluster pro
Creating a custom Dataproc Image Google provides an option to launch a Dataproc cluster with a custom machine image Custom machine image will decre...
Tue, 15 Nov, 2022 at 1:44 AM
Steps to Update Databricks Token on IWX Version 5.2.0
This Article has the steps document to update Databricks token IWX Version 5.2.0 Instructions Prerequisites This script must be run on a VM becaus...
Wed, 24 Aug, 2022 at 7:34 PM
How to add labels for Dataproc Ephemeral clusters launched by Ingestion jobs?
Problem: I would like to add labels to Dataproc Cluster launched by Infowokrs ingestion jobs Solution: Infoworks provides a pre ingestion jo...
Fri, 13 May, 2022 at 9:21 AM
How to update the default autoscaling policy for Dataproc Ephemeral clusters ?
Problem: I would like to use a custom autoscaling policy for my Dataproc Cluster for Ephemeral jobs or I would like to use secondary worker nodes for th...
Wed, 11 May, 2022 at 8:28 PM
How to use custom user defined functions in pipelines?
Problem Description: I would like to use a custom UDF in Infoworks Pipelines Solution: You could use custom user-defined functions after regi...
Wed, 11 May, 2022 at 12:02 AM
Delegation token expiry
Infoworks uses a JWT token for internal services communication, this token has to be updated by March 8th, 2022 Effected Infoworks Versions: 2.9.x,3....
Wed, 18 May, 2022 at 1:08 PM
Throttling exception in High-level application history section
EMR Has a limitation in displaying the relevant application job links in High-level application history section as evidenced in the attached screenshot ...
Fri, 7 Jan, 2022 at 1:06 AM
csv_error_threshold in 3.x is changed to error_record_threshold in 5.0
Problem statement: In earlier versions of Infoworks3.x,4.x we had csv_error threshold as advance config for setting up the threshold for CSV error records ...
Thu, 6 Jan, 2022 at 7:16 PM
Steps to enable log rotation in IWX 5.x
Problem Description:  How to enable log rotation in IWX 5.x Solution: Login into the Infoworks edge node as the user who starts IWX services execute ...
Wed, 22 Dec, 2021 at 12:19 AM
Enabling log rotation fails in Infoworks v5.0/v5.1
Problem Description: Enabling log rotation fails with the below error in Infoworks v5.0/v5.1.1 Enabling log rotation Connected to MongoDB... Connected t...
Tue, 23 Nov, 2021 at 10:27 PM