How to send Spark bank to the EMR cluster?

Question

I used the AWS online console to start my cluster with Apache Spark. I have a thick jar based on my Spark app and I loaded it into an S3 bucket. When I try to send it as a Step with a Custom Jar , the process fails. Any pointers would be greatly appreciated.

Answer 1

Use the EMR boot hopper to install Spark and submit the task as described in the documentation: https://github.com/awslabs/emr-bootstrap-actions/blob/master/spark/examples/spark-submit-via-step.md