Runtime Processing and Time Secrecy in AWS CodePipeline

Question

We are dealing with the problem of providing assembly time and runtime of our applications created using AWS CodePipeline, and their deployment in ECS.

Ultimately, our vision is to create a common pipeline for each of our applications, which achieves the following goals:

• Full access sharing
  • Services in the pipeline application CANNOT access any credentials or use any of the keys used in the app-b pipeline and vice versa.
• Secret management of designated developers
  • Only developers responsible for application-a can read and write secrets for application-a

Here are the problems:

• Some of our applications require access to private repositories to resolve dependencies during build. For example, our Java applications require access to a private maven repository for successful builds.
• Some of our applications require database access credentials at runtime. For example, the servlet container on which our application is running requires a .xml configuration file that contains credentials for searching and accessing databases.

Along with some reservations:

• Our code base is in an open repository. We do not want to disclose secrets by placing either plaintext or the secret text of this cache in our repository.
• We don’t want to bake run-time secrets in our Docker images created in CodeBuild, even if access to ECR is limited.
• The Cloudformation template for ECS resources and the associated parameter file are located in the open repository in clear text. This eliminates the possibility of passing runtime secrets to the ECS cloud statistics template via parameters (as I understand it)

We looked at using tools like credstash to help with credential management. This solution requires both instances of the CodeBuild and ECS tasks to be able to use the AWS CLI. To avoid shuffling more credentials, we decided it was best to assign privileged roles to instances that require the use of the AWS CLI. That way, the CLI can pull credentials out of a role in instance metadata

We tried to develop a way to manage our secrets, given these limitations. For each application, we create a pipeline. Using the Cloudformation template, we create:

• 4 resources:

  • DynamoDB Credential Table
  • KMS Account Key
  • ECR Reservation
  • CodePipeline (creation, deployment, etc.)

• 3 roles:

  • CodeBuildRole Read Access to DynamoDB Credential Table Decrypt Permission Using KMS Key Write to ECR Repository
  • ECSTaskRole Read access to credential table DynamoDB Decrypt permission using KMS key Read from ECR repository
  • DeveloperRole Read and write access to the credential table DynamoDB Encryption and decryption using the KMS key

The CodeBuild step in CodePipeline assumes that CodeBuildRole allows you to read build time secrets from a credential table. CodeBuild then creates the project and generates a Docker image that it clicks on the ECR. In the end, the deployment step creates the ECS service using the cloud information template and the related parameters file present in the common project repository. The ECS task definition includes the adoption of ECSTaskRole to allow tasks to read run-time secrets from the credential table and pull the desired image from ECR.

Here is a simple diagram of AWS resources and their relationships as described above

Our current proposed solution has the following problems:

• The role is heavy
  • Creating roles is a privileged activity in our organization. Not all developers who are trying to create the above pipeline will receive permission to create the necessary roles.
• Intended DeveloperRole Developer Guide:
  • According to these developers, you must manually accept the DeveloperRole. We played with the idea of ​​moving to the ARN list of developer users as a parameter of the Cloudformation pipeline template. Does Cloudformation have a mechanism for assigning a role or policy to a specified user?

Is there a more established way to pass secrets to CodePipeline that we can ignore, or is this the best we can get?

Answer 1

Two thoughts:

• AWS Options Storage
• IAM roles for Amazon ECS tasks

The AWS settings repository can protect access keys with extended access. This access can be based on ServiceRoles.

ECS provides access to ServiceRole through this template:

build: commands: - curl 169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI | jq 'to_entries | [ .[] | select(.key | (contains("Expiration") or contains("RoleArn")) | not) ] | map(if .key == "AccessKeyId" then . + {"key":"AWS_ACCESS_KEY_ID"} else . end) | map(if .key == "SecretAccessKey" then . + {"key":"AWS_SECRET_ACCESS_KEY"} else . end) | map(if .key == "Token" then . + {"key":"AWS_SESSION_TOKEN"} else . end) | map("export \(.key)=\(.value)") | .[]' -r > /tmp/aws_cred_export.txt - chmod +x /tmp/aws_cred_export.txt - /aws_cred_export.txt && YOUR COMMAND HERE 

If your ServiceRole provided for the CodeBuild task has access to use the parameter save key, you should be kind.

Happy hunting and hope this helps

Answer 2

At a high level, you can isolate applications in a single AWS account using granular permissions (this is similar to what you are trying to do) or using multiple AWS accounts. None of these are right or wrong as such, but I prefer separate AWS accounts to manage granular permissions because your starting point is complete isolation.