All geek questions in one place

Stuck in WAITING_FOR_RUNNER when using AMI for EC2 Resource

I was able to successfully execute the data pipeline for predefined templates. I wanted to use the configured AMI for my EC2 resource, as I need to install some libraries and packages.

I also need to run a python script as part of the process.

So, I created a basic EC2 Resource image with all the necessary packages and code that needs to be run.

As part of my activity, I run the shell command action, where I run the python script as the command to be run.

The EC2 resource starts successfully based on the configured AMI that I specified. I can enter this machine using the key pair that I specified, but only that the Activity was stuck in the "WAIT FOR RUNNER" state.

I am not sure how to solve this problem. Please let me know if there are any fixes. I skipped some basic step while trying to use AMI's EC2 resource.

+4
source share
3 answers

Use Amazon Linux when creating a custom AMI, and it will solve this problem. Their OS comes preloaded with tools that are used by data pipelines to communicate with the instance.

+2
source

Do you work in VPC or EC2Classic? I had the same problem when working in VPC. When I checked run.out on an EC2 instance, I saw an error message:

Error in custom provider, java.lang.RuntimeException: java.net.UnknownHostException:., "

TaskRunner was unable to resolve its own hostname and did not start.

I solved this by setting the "DNS hostnames" parameter to yes on my VPC in the VPC console. By default, new accounts are set to no. This solved the problem.

0
source

I understand that this is old, but if you are using a custom AMI with the specified runson, you must make sure that your custom AMI has all the dependencies installed: https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp- custom-ami.html

0
source

Source: https://habr.com/ru/post/1493418/

More articles:


All Articles