How-to

How to set ERROR record threshold value for CSV ingestion in Infoworks DataFoundry
Description: The below configuration allows the CSV ingestion job to have some error records as configured and DataFoundry would not fail the job. The inges...
Thu, 6 Jan, 2022 at 7:24 PM
How to add a timestamp column from regex on filename during file ingestion?
Datafoundry fixed-width ingestion provides an option to append new columns which are extracted from a "match regex" and "extract regex" ...
Mon, 24 Aug, 2020 at 9:44 PM
How to extract filename or part of file name using Data Transformation?
As an example below are the instructions to ingest a file with file name appended with a timestamp and use the Datafoundry pipeline to create a table with a...
Mon, 24 Aug, 2020 at 9:45 PM
How to configure SSL for DataFoundry using a PFX format SSL certificate?
Problem Description:  Infoworks DataFoundry service requires SSL certificate and private key as separate files to configure SSL encryption Many SSL ven...
Fri, 10 Jul, 2020 at 2:40 AM
Steps to perform Python Custom Transformation in Infoworks
Scenario: If a dataset like the one mentioned below: id,fname,lname,salary 1,Nitin,BS,10 2,Alex,P,20 3,Hrithik,R,25 And, the requirement is like he...
Wed, 27 Oct, 2021 at 5:10 PM
HOW TO: Create a Signed SSL certificate to enable SSL in Infoworks
This Solution article lists the steps to create a signed certificate using the CSR. 1. Create your private key and the Certificate Signing Request. ...
Thu, 13 Feb, 2020 at 10:28 PM
How to read the ORC/Parquet files content in HDFS
Description: Some times we might need to read customers ORC/Parquet file content while debugging issues related to special characters/ encoding related ...
Tue, 28 May, 2019 at 1:21 PM
How to access the Big query audit logs
Description: Some times we need to access the Big Query audit logs to debug issues related to Big query export in Infoworks ADE. We can use bq command l...
Wed, 15 May, 2019 at 12:22 PM