Steps to perform rest ingestion using cdata rest driver.
1) Download the cdata rest driver jar from the cdata website by following the below solution article.
(Note: Ensure that you upload all the below mentioned files in the screenshot while getting the Driver extension created on infoworks)
2) Once you have the driver ready, Create the Cdata rest source providing the below mentioned details in the screenshot.
Generate the refresh token( One timer during the initial configuration) by running the below curl command:
curl -k -d 'email@example.com&password=###########&scope=offline_access+MiX.Integrate' -H 'Authorization: Basic ##############Q=' -H 'Content-Type: application/x-www-form-urlencoded' https://identity.us.mixtelematics.com/core/connect/token
The result would look like the one below:
Use the above refresh token while configuring the source.
3) Ensure that you pass the OAuthSettingLocation parameter which is last parameter in the above image and point it to a location on the Edgenode.
4) Run Save and Test Connection. Ensure that the connection is successful. This would create OAuthSettings.txt in the above location specified by you on the edgenode.
Once you have the OAuth settings file, you would need to move that file to dbfs location.
From edgenode run the below command to move the file to dbfs:
Eg: dbfs cp /home/infoworks/OAuthSettings.txt
5) Create an init script with name lets say init_auth.sh
cp /dbfs/FileStore/tables/nitin/OAuthSetting2.txt /home/infoworks/OAuthSetting2.txt;
ls -ltrh /home/infoworks/OAuthSetting2.txt
Ensure that while providing the above copy command, the source location points to your dbfs OAuthSettings.txt file.
And destination location is same as what you provided the value for OAuthSettingsLocation parameter while creating the source.
5.5) Copy the init_auth.sh script to dbfs
dbfs cp ./init_auth_mix.sh dbfs:/infoworks/lib/scripts/init_auth.sh
6) Open /opt/infoworks/conf/databricks_defaults.json on the edgenode.
And search for the init_scripts key word.
And update the value as shown below:
If there are entries already present in the init_scripts, just add the below entry to the array.
This would allow all the job cluster launched by Infoworks jobs to be launched with the access to
7) Run the Ingestion Job and validate the data on the target.
Applicable versions: 4.2