At times end users might need to tune certain Spark configurations to support certain workloads or launch clusters with additional spark configurations.
On Infoworks version 5.2 and above, the custom Spark configuration can be configured through the Cluster template.
1. Navigate to Admin > Manage Enviornaments > Choose the Environment > Click on the Compute tab
2. Expand the 'Advanced Configuration' section and click on 'Add'
Value: (semicolon separated configuration key-value pairs)
Spark configurations can also be passed as arguments for individual jobs such as ingestion, sync to target, and pipeline