How-to

Add a new streaming extension to configure the custom deserializer for Kafka ingestion
Note: The provided custom deserializer JAR is designed specifically for writing a single JSON message from a Kafka topic to a single column in target table....
Thu, 7 Dec, 2023 at 7:49 PM
How to add Infoworks audit columns to custom target tables
______________________________________________________________________________________________________ We have Two Options to include ZIW Columns in the p...
Thu, 26 Oct, 2023 at 7:33 AM
How to configure custom Spark settings at Cluster template
At times end users might need to tune certain Spark configurations to support certain workloads or launch clusters with additional spark configurations.  ...
Tue, 15 Nov, 2022 at 4:16 AM
How to add a new region for AWS Databricks in Infoworks
To add a new region version for AWS Databricks, perform the following steps: Step 1: SSH into Infoworks machine. Step 2: Navigate to the path where I...
Fri, 24 Jun, 2022 at 1:22 PM
SAML integration with Okta
Fri, 24 Jun, 2022 at 5:25 AM
How to get the tpt.out file for a failed TPT ingestion job in IWX 5.0
Description: During the Teradata TPT ingestion, Infoworks DataFoundry would use the Teradata Parallel Transport Utility installed on the Master node of the ...
Wed, 15 Sep, 2021 at 3:27 PM
How to debug and resolve pipelines going to blocked state in AFLAC Prod Env
Description: Pipelines in AFLAC would go to Blocked state as the MongoDB service on the IWX Edge node crashes because of the compatibility issues of MongoDB...
Tue, 5 Oct, 2021 at 10:16 PM
How to remove duplicate records in full load ingestion?
Description: Set the below-advanced configuration to remove duplicate records during full load ingestion. This can be set to either the source or at table l...
Mon, 9 Aug, 2021 at 1:35 PM
How to map existing EMR/Dataproc cluster to Persistent compute
Infoworks 5.0 versions provides the ability to launched persistent cluster from Infoworks UI. However if one wants to map an existing cluster to infoworks p...
Fri, 22 Mar, 2024 at 12:42 PM
How to take backup of Infoworks Postgres DB
Description: Perform the below steps to take back up of Infoworks Postgres DB. i) source /opt/infoworks/bin/env.sh Assuming that Infoworks is instal...
Mon, 12 Jul, 2021 at 2:29 PM