Problem Description:


Spark pipeline build job fails with then below SASL Authentication error in the job log in Infoworks DataFoundry EMR 3.1.1.


Failed to connect to external shuffle server, will retry 1 more time after waiting 5 seconds...

java.lang.RuntimeException: java.lang.IllegalStateException: Expected SaslMessage, received something else (maybe your client does not have SASL enabled?)


Root cause:


This issue happens if Network Encryption is enabled from the EMR Cluster end but the –conf spark.authenticate=true” is not passed in the Spark job conf.

https://www.ericlin.me/2018/06/spark-job-sasl-authentication-error/

Solution: Perform the below steps and run the Spark pipeline build job again.


a) Login to Infoworks DataFoundry edge node as Infoworks user.

b) Go to $IW_HOME/conf directory.

c) vi dt_spark_defaults.conf

d) Add the below properties, save the file and run the pipeline job.


     spark.authenticate true


Applicable Infoworks DataFoundry Versions:

v3.1.x