Problem Description


XML based rest API ingestion fails with below error


Caused by: java.util.concurrent.ExecutionException: Boxed Error
at scala.concurrent.impl.Promise$.resolver(Promise.scala:55) 
at scala.concurrent.impl.Promise$.scala$concurrent$impl$Promise$$resolveTry(Promise.scala:47)
at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:244)
at scala.concurrent.Promise$class.tryFailure(Promise.scala:112)
at scala.concurrent.impl.Promise$DefaultPromise.tryFailure(Promise.scala:153)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:739)
Caused by: java.lang.NoClassDefFoundError: org/json/XML
at consumers.genericrest.GenericRestUtility.convertXMLTOJSON(GenericRestUtility.java:811)
at consumers.genericrest.GenericRestUtility.getJsonRecord(GenericRestUtility.java:790)
at consumers.genericrest.GenericRestUtility.traversePages(GenericRestUtility.java:371)
at consumers.genericrest.GenericRestUtility.copyPaginationDataToHDFS(GenericRestUtility.java:339)
at consumers.genericrest.GenericRestUtility.copyRestDataToHDFS(GenericRestUtility.java:648)
at consumers.genericrest.GenericRestSparkConsumer.run(GenericRestSparkConsumer.java:49)
at consumers.genericrest.GenericRestSparkDriver.main(GenericRestSparkDriver.java:45)


Cause 


This happens if the required JSON jar is not present in the spark_streaming_job_classpath in conf.properties.


Solution


a) Login to the edge node as a user who starts Infoworks Services.

b) Download the latest JSON jar using the below command.


wget https://repo1.maven.org/maven2/org/json/json/20200518/json-20200518.jar


c)Take the backup of conf.properties file in $IW_HOME/conf directory.

d) Add the absolute path of downloaded JSON jar at the end in the spark_streaming_job_classpath in conf.properties.

e) Restart the platform service.


Affected Versions


V3.1-GCP