AWS CLI creates RDS with an environment

Question

How to create an RDS instance using create-environment or another aws elasticbeanstalk subcommand? I tried several combinations of parameters to no avail. The following is an example.

 APP_NAME="randall-railsapp" aws s3api create-bucket --bucket "$APP_NAME" APP_VERSION="$(git describe --always)" APP_FILE="deploy-$APP_NAME-$APP_VERSION.zip" git archive -o "$APP_FILE" HEAD aws s3 cp "$APP_FILE" "s3://$APP_NAME/$APP_FILE" aws --region us-east-1 elasticbeanstalk create-application-version \ --auto-create-application \ --application-name "$APP_NAME" \ --version-label "$APP_VERSION" \ --source-bundle S3Bucket="$APP_NAME",S3Key="$APP_FILE" aws --region us-east-1 elasticbeanstalk create-environment \ --application-name "$APP_NAME" \ --version-label "$APP_VERSION" \ --environment-name "$APP_NAME-env" \ --description "randall rails app environment" \ --solution-stack-name "64bit Amazon Linux 2014.03 v1.0.0 running Ruby 2.1 (Puma)" \ --cname-prefix "$APP_NAME-test" \ --option-settings file://test.json 

And the contents of test.json :

 [ { "OptionName": "EC2KeyName", "Namespace": "aws:autoscaling:launchconfiguration", "Value": "a-key-is-here" }, { "OptionName": "EnvironmentType", "Namespace": "aws:elasticbeanstalk:environment", "Value": "SingleInstance" }, { "OptionName": "SECRET_KEY_BASE", "Namespace": "aws:elasticbeanstalk:application:environment", "Value": "HAHAHAHAHAHA" }, { "OptionName": "DBPassword", "Namespace": "aws:rds:dbinstance", "Value": "hunter2" }, { "OptionName": "DBUser", "Namespace": "aws:rds:dbinstance", "Value": "random" }, { "OptionName": "DBEngineVersion", "Namespace": "aws:rds:dbinstance", "Value": "9.3" }, { "OptionName": "DBEngine", "Namespace": "aws:rds:dbinstance", "Value": "postgres" } ] 

Does anyone know why this fails? Everything I specify in the aws:rds:dbinstance seems to be removed from the configuration.

Answer 1

Just setting the aws: rds: dbinstance parameters does not create an RDS database. You can currently create an instance of RDS using one of the following methods:

• Create with AWS Console
• Use eb cli
• Use the ebextensions resource section to create an RDS resource

The first two approaches are most convenient, since they do all the hard lifting for you, but for the third you have to do extra work. The third approach is what you would like to use if you are not using the console or eb CLI.

You can create an RDS resource for your beanstalk environment using the following ebextension snippet. Create a file called 01-rds.config in the .ebextensions directory of your application source.

 Resources: AWSEBRDSDatabase: Type: AWS::RDS::DBInstance Properties: AllocatedStorage: 5 DBInstanceClass: db.t2.micro DBName: myawesomeapp Engine: postgres EngineVersion: 9.3 MasterUsername: myAwesomeUsername MasterUserPassword: myCrazyPassword 

This file is in YAML format, so it is important to indent. You can also use JSON if you want. These are not parameter settings, so you cannot pass them as --option-settings test.json . You just need to link this file to your application source.

Read more about the properties that you can configure in your RDS database here . On this page you can also find which properties are required and which properties are optional.

Let me know if the above does not work for you or if you have additional questions.

Answer 2

I had the same problem, I could not get it to work through .ebextensions, and I do not like the EB CLI tool.

The EB CLI uses an undocumented API function and a custom version of the botocor library ("eb_botocore") to do this job. :(

So, I went ahead and forked the botocore and combined in the API data file used by eb_botocore: https://github.com/boto/botocore/pull/396

Then I ran "python setup.py install" on my modified bot cord and aws-cli (like the wizard), and aws-cli now accepts the -template-specification option in the aws elasticbeanstalk create-environment command. Hooray!

Usage example:

 aws elasticbeanstalk create-environment\ ...various options...\ --option-settings file://option-settings.json --template-specification file://rds.us-west-2.json 

where rds.us-west-2.json:

 { "TemplateSnippets": [{ "SnippetName": "RdsExtensionEB", "Order": 10000, "SourceUrl": "https://s3.amazonaws.com/elasticbeanstalk-env-resources-us-west-2/eb_snippets/rds/rds.json" }] } 

(it seems you should select a fragment specific to your EB region).

and option-settings.json contains RDS-related parameters similar to those listed in the question (DBEngine, DBInstanceClass, DBAllocatedStorage, DBPassword).

It works great. I hope the AWS CLI team will allow us to use this feature in an official tool in the future. I suppose this is not a trivial change, or they would have done it already, but this is a pretty serious omission of functionality from the Elastic Beanstalk API and AWI CLI APIs, so I hope they crack it.

Answer 3

Other answers did not work in my environment as of September 2015. After much trial and error, the following worked for me:

template configuration fragment (YAML):

  aws:rds:dbinstance: DBAllocatedStorage: '5' DBDeletionPolicy: Delete DBEngine: postgres DBEngineVersion: 9.3.9 DBInstanceClass: db.t2.micro DBPassword: PASSWORD_HERE DBUser: USERNAME_HERE MultiAZDatabase: false 

.ebextensions / rds.config file (JSON):

 { "Parameters": { "AWSEBDBUser": { "NoEcho": "true", "Description": "The name of master user for the client DB Instance.", "Default": "ebroot", "Type": "String", "ConstraintDescription": "must begin with a letter and contain only alphanumeric characters" }, "AWSEBDBPassword": { "NoEcho": "true", "Description": "The master password for the DB instance.", "Type": "String", "ConstraintDescription": "must contain only alphanumeric characters" }, "AWSEBDBName": { "NoEcho": "true", "Description": "The DB Name of the RDS instance", "Default": "ebdb", "Type": "String", "ConstraintDescription": "must contain only alphanumeric characters" } }, "Resources": { "AWSEBAutoScalingGroup": { "Metadata": { "AWS::ElasticBeanstalk::Ext": { "_ParameterTriggers": { "_TriggerConfigDeployment": { "CmpFn::Insert": { "values": [ { "CmpFn::Ref": "Parameter.AWSEBDBUser" }, { "CmpFn::Ref": "Parameter.AWSEBDBPassword" }, { "CmpFn::Ref": "Parameter.AWSEBDBName" } ] } } }, "_ContainerConfigFileContent": { "plugins": { "rds": { "Description": "RDS Environment variables", "env": { "RDS_USERNAME": { "Ref": { "CmpFn::Ref": "Parameter.AWSEBDBUser" } }, "RDS_PASSWORD": { "Ref": { "CmpFn::Ref": "Parameter.AWSEBDBPassword" } }, "RDS_DB_NAME": { "Ref": { "CmpFn::Ref": "Parameter.AWSEBDBName" } }, "RDS_HOSTNAME": { "Fn::GetAtt": [ "AWSEBRDSDatabase", "Endpoint.Address" ] }, "RDS_PORT": { "Fn::GetAtt": [ "AWSEBRDSDatabase", "Endpoint.Port" ] } } } } } } } }, "AWSEBRDSDatabase": { "Type": "AWS::RDS::DBInstance", "DeletionPolicy": "Delete", "Properties": { "DBName": { "Ref": { "CmpFn::Ref": "Parameter.AWSEBDBName" } }, "AllocatedStorage": "5", "DBInstanceClass": "db.t2.micro", "Engine": "postgres", "DBSecurityGroups": [ { "Ref": "AWSEBRDSDBSecurityGroup" } ], "MasterUsername": { "Ref": { "CmpFn::Ref": "Parameter.AWSEBDBUser" } }, "MasterUserPassword": { "Ref": { "CmpFn::Ref": "Parameter.AWSEBDBPassword" } }, "MultiAZ": false } }, "AWSEBRDSDBSecurityGroup": { "Type": "AWS::RDS::DBSecurityGroup", "Properties": { "DBSecurityGroupIngress": { "EC2SecurityGroupName": { "Ref": "AWSEBSecurityGroup" } }, "GroupDescription": "Enable database access to Beanstalk application" } } } } 

Answer 4

As of December 2017 we use the following ebextensions

 $ cat .ebextensions/rds.config Resources: AWSEBRDSDBSecurityGroup: Type: AWS::RDS::DBSecurityGroup Properties: EC2VpcId: Fn::GetOptionSetting: OptionName: "VpcId" GroupDescription: RDS DB VPC Security Group DBSecurityGroupIngress: - EC2SecurityGroupId: Ref: AWSEBSecurityGroup AWSEBRDSDBSubnetGroup: Type: AWS::RDS::DBSubnetGroup Properties: DBSubnetGroupDescription: RDS DB Subnet Group SubnetIds: Fn::Split: - "," - Fn::GetOptionSetting: OptionName: DBSubnets AWSEBRDSDatabase: Type: AWS::RDS::DBInstance DeletionPolicy: Delete Properties: PubliclyAccessible: true MultiAZ: false Engine: mysql EngineVersion: 5.7 BackupRetentionPeriod: 0 DBName: test MasterUsername: toor MasterUserPassword: 123456789 AllocatedStorage: 10 DBInstanceClass: db.t2.micro DBSecurityGroups: - Ref: AWSEBRDSDBSecurityGroup DBSubnetGroupName: Ref: AWSEBRDSDBSubnetGroup Outputs: RDSId: Description: "RDS instance identifier" Value: Ref: "AWSEBRDSDatabase" RDSEndpointAddress: Description: "RDS endpoint address" Value: Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Address"] RDSEndpointPort: Description: "RDS endpoint port" Value: Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Port"] AWSEBRDSDatabaseProperties: Description: Properties associated with the RDS database instance Value: Fn::Join: - "," - - Ref: AWSEBRDSDatabase - Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Address"] - Fn::GetAtt: ["AWSEBRDSDatabase", "Endpoint.Port"] 

With such custom options

 $ cat .ebextensions/custom-options.config option_settings: "aws:elasticbeanstalk:customoption": DBSubnets: subnet-1234567,subnet-7654321 VpcId: vpc-1234567 

One thing: you must explicitly pass the env RDS_ * variables to your application.