I decided my own question, but still send it in the hope of saving someone else a few hours!
I have an AWS server project using Python to insert an entry into the kinesia queue. However, when I use boto3.client ('kinesis') or the put_record function, it seems to freeze until the time runs out, with no error messages or other information. Below is the function:
import boto3 def put_record_kinesis(data, stream_name, partition_key): print "create kinesis begin" kinesis = boto3.client("kinesis") print "put record begin" response = kinesis.put_record(StreamName=stream_name, Data=data, PartitionKey=partition_key) print "put record complete" print response
The serverless.yml definition is as follows:
provider: name: aws runtime: python2.7 iamRoleStatements: - Effect: "Allow" Action: - "ec2:CreateNetworkInterface" - "ec2:DescribeNetworkInterfaces" - "ec2:DeleteNetworkInterface" - "kinesis:*" Resource: "*" vpc: securityGroupIds: - sg-... subnetIds: - subnet-... - subnet-... - subnet-... stage: dev region: eu-west-1 memorySize: 128 functions: LambdaQueueFunction: handler: python_file.queue memorySize: 1024 timeout: 100 LambdaDequeueFunction: handler: python_file.dequeue resources: Resources: KinesisQueue: Type: AWS::Kinesis::Stream Properties: Name: kinesis-queue ShardCount: 1 ChronosQueueMap: Type: AWS::Lambda::EventSourceMapping DependsOn: - "LambdaDequeueFunctionLambdaFunction" - "IamPolicyLambdaExecution" Properties: BatchSize: 1 EventSourceArn: Fn::GetAtt: - "KinesisQueue" - "Arn" FunctionName: Fn::GetAtt: - "LambdaDequeueFunctionLambdaFunction" - "Arn" StartingPosition: "TRIM_HORIZON"
When I run the function, I see the following in the cloud observation logs:
10:53:02 | START RequestId: 027bb0cb-acb4-11e6-b20c-1b587b734943 Version: $LATEST 10:53:02 | put records begin 10:54:42 | END RequestId: 027bb0cb-acb4-11e6-b20c-1b587b734943 10:54:42 | REPORT RequestId: 027bb0cb-acb4-11e6-b20c-1b587b734943 Duration: 100002.99 ms Billed Duration: 100000 ms Memory Size: 1024 MB Max Memory Used: 22 MB 10:54:42 | 2016-11-17T10:54:42.155Z 027bb0cb-acb4-11e6-b20c-1b587b734943 Task timed out after 100.00 seconds
It turns out the solution was that the lambda function did not have access to the Internet. By default, the non-VPC lambda function has Internet access, but the lambda function inside the VPC does not.
To fix this, I created a new subnet, a route table, an elastic IP, and a gateway. They were configured as follows:
- The native gateway uses an elastic IP and points to any subnet with an Internet gateway.
- The route table has a route for local traffic (..0.0 / 16 | Local | Active) and a route for all other IP addresses to the nat gateway (0.0.0.0/0 | NAT ID | Active)
- It is established that a new route table is being used.
Hope this helps someone!