Amazon ECS - persistent data between instances

Question

What's the best way to handle persistent data between instances using a load-balanced service in Amazon ECS? Data containers will not work, and not volumes that you can specify in tasks, they will be stored only on the instance itself. I am trying to read about installing EBS when creating an instance using User Data in Launch Configuration, but I was out of luck there.

Answer 1

Depending on your data needs, you have two options that I can think of:

Mapping S3 Bucket as Local Disk

You can split the S3 bucket and restrict access to any number of instances. We are using a drive mapping solution on Windows that will mount the S3 bucket as a local drive. Similar drivers exist for Linux . Thus, each instance receives the same mapped drive and transfers this persistent data. Data is read / written, so if we scale or exit, each instance has access to S3 data in a consistent format.

Mount the volume from the snapshot

If you think this is readable data that you need access to, you can use Userdata scripts to mount the volume from the snapshot at startup. You just need a script and an IAM account / role to run the appropriate commands at startup

Answer 2

You can use Amazon EFS to share the file system through containers and ECS instances. EFS is based on NFS, so it can be installed on multiple host instances at the same time. This allows you to work with planning and scaling the cluster. See the MySQL Data Retention Guide as follows:

https://aws.amazon.com/blogs/compute/using-amazon-efs-to-persist-data-from-amazon-ecs-containers/

Answer 3

I suggest using Amazon EFS ( https://aws.amazon.com/blogs/compute/using-amazon-efs-to-persist-data-from-amazon-ecs-containers/ ).

Just add the restriction that there are only 4 regions to support EFS.

EU (Ireland)

US East (N. Virginia)

US East (Ohio)

West United States (Oregon)

If your region is not supported, we can implement our own NFS resource to share a persistent folder between EC2 instances. S3FS looks cool, but it's a buggy (I tested 2 years ago. Today may change)