Description


By default the Infoworks Job clusters spun up for ingestion or pipelines jobs will have a few default tags. 

If customers wish to add a few custom tags to the Job clusters please follow this document.


Solution


We can add custom tags by editing the file {$IW_HOME}/conf/databricks_defaults.json in the Infoworks Edge node.

Example:

From
"standard": {
"custom_tags": {},
"spark_env_vars": {
"PYSPARK_PYTHON": "/databricks/python3/bin/python3"
},  
 to
"standard": {
"custom_tags": {"environment":"demo", "product":"4.2iwx"},
"spark_env_vars": {
"PYSPARK_PYTHON": "/databricks/python3/bin/python3"
},


Note: Databricks allows at most 45 custom tags


Application versions


All versions of Datafoundry on Databricks