Infoworks

Steps to perform rest ingestion using cdata rest driver.

1) Download the cdata rest driver jar from the cdata website by following the below solution article.

(Note: Ensure that you upload all the below mentioned files in the screenshot while getting the Driver extension created on infoworks)

https://infoworks.freshdesk.com/a/solutions/articles/14000123964

2) Once you have the driver ready, Create the Cdata rest source providing the below mentioned details in the screenshot.

Generate the refresh token( One timer during the initial configuration) by running the below curl command:

curl -k -d 'grant_type=password&username=xxxxx@xxx.com&password=###########&scope=offline_access+MiX.Integrate' -H 'Authorization: Basic ##############Q=' -H 'Content-Type: application/x-www-form-urlencoded'  https://identity.us.mixtelematics.com/core/connect/token

The result would look like the one below:

{"access_token":"##########","expires_in":3600,"token_type":"Bearer","refresh_token":"1234567891234456787654"}

Use the above refresh token while configuring the source.

3) Ensure that you pass the OAuthSettingLocation parameter which is last parameter in the above image and point it to a location on the Edgenode.

Eg: /home/infoworks/OAuthSettings.txt

4) Run Save and Test Connection. Ensure that the connection is successful. This would create OAuthSettings.txt in the above location specified by you on the edgenode.

Once you have the OAuth settings file, you would need to move that file to dbfs location.

From edgenode run the below command to move the file to dbfs:

Eg: dbfs cp /home/infoworks/OAuthSettings.txt 
dbfs:/FileStore/tables/nitin/OAuthSetting2.txt

5) Create an init script with name lets say init_auth.sh

With contents:

#!/bin/bash

mkdir /home/infoworks/

cp /dbfs/FileStore/tables/nitin/OAuthSetting2.txt /home/infoworks/OAuthSetting2.txt;

ls -ltrh /home/infoworks/OAuthSetting2.txt

Ensure that while providing the above copy command, the source location points to your dbfs OAuthSettings.txt file.

Eg: /dbfs/FileStore/tables/nitin/OAuthSetting2.txt 

And destination location is same as what you provided the value for OAuthSettingsLocation parameter while creating the source.

Eg:/home/infoworks/OAuthSetting2.txt

5.5) Copy the init_auth.sh script to dbfs

dbfs cp ./init_auth_mix.sh dbfs:/infoworks/lib/scripts/init_auth.sh

6) Open /opt/infoworks/conf/databricks_defaults.json on the edgenode.

And search for the init_scripts  key word.

And update the value as shown below:

If there are entries already present in the init_scripts, just add the below entry to the array.

"init_scripts": [
        {
        "dbfs": {
        "destination":"dbfs:${databricks_iw_home}/lib/scripts/init_auth.sh"
       }
    }
  ]

This would allow all the job cluster launched by Infoworks jobs to be launched with the access to 

OAuthSettings.txt file.

7) Run the Ingestion Job and validate the data on the target.

Applicable versions: 4.2