Infoworks

This document describes the services that are a part of the Infoworks application and their corresponding log location: User Interface (UI):  This Compon...

Below is a script to monitor for active EMR job drivers that start with "ClusterJobDriver_". At any given time there should be only one "Clus...

Problem Statement: Whenever Sync to target to Teradata fails with the below error because of TERADATA FASTLOAD. This is the Fastload issue. Error: Cau...

# Script to compare the same files in 2 directories.  # List of files that need to be compared needs to passed as a separate file # The script takes input...

Scenario 1: Row count for the table replicated on dataproc hive is not matching with row count of the table in on-premise HDP hive. When user submits a &qu...

Problem statement: Pipeline using replicated tables as source tables may fail with below error similar to the show below. Caused by: java.lang.ClassCastE...

Creating a custom Dataproc Image Google provides an option to launch a Dataproc cluster with a custom machine image Custom machine image will decre...

This Article has the steps document to update Databricks token IWX Version 5.2.0 Instructions Prerequisites This script must be run on a VM becaus...

Problem: I would like to add labels to Dataproc Cluster launched by Infowokrs ingestion jobs Solution: Infoworks provides a pre ingestion jo...

Problem: I would like to use a custom autoscaling policy for my Dataproc Cluster for Ephemeral jobs or I would like to use secondary worker nodes for th...