Amazon Aurora 1.8 Downloading data from S3 - cannot start S3 client

Question

In the last update of Aurora (1.8), a command was introduced LOAD DATA FROM S3. Has anyone got this to work? After upgrading to 1.8, I followed the installation guide here to create a role to allow access from RDS to S3.

After rebooting the server and trying to run the command

LOAD DATA FROM S3 PREFIX 's3://<bucket_name>/prefix' INTO TABLE table_name

in SQL Workbench / J I get errors:

Warnings:
S3 API returned error: Missing Credentials: Cannot instantiate S3 Client
S3 API returned error: Failed to instantiate S3 Client
Internal error: Unable to initialize S3Stream

Are there any additional steps needed? Can I only run this from the SDK? I do not see anywhere mentioned in the docs

Answer 1

. AmazonS3FullAccess IAM, RDS... .

, , RDS Clusters. Aurora " IAM". , IAM (, ).

, , .

, ( ) 5 /:

1) S3,

{
    "Version": "2012-10-17",
    "Id": "Policy1453918146601",
    "Statement": [
        {
            "Sid": "Stmt1453917898368",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<account id>:<user/group/role>/<IAM User/Group/Role>"
            },
            "Action": [
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::<bucket name>/*"
        }
    ]
}

"" , IAM, , RDS .

2) IAM:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1486490368000",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:GetObjectVersion",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::<bucket name>/*"
            ]
        }
    ]
}

.

3) IAM:

IAM . , , , , , .

4) , / aws_default_s3_role ARN # 3 .

5) Aurora, , , " IAM" IAM DB

, .

, !

Answer 2

, , VPC S3. .

Answer 3

2019 :

RDS . CLI, .

aws rds add-role-to-db-cluster --db-cluster-identifier my-cluster --role-arn arn:aws:iam::123456789012:role/AllowAuroraS3Role

Answer 4

Internal error: Unable to initialize S3Stream, , :

• ,
• , Amazon S3: s3-region://-/ ---

:

• () - AWS, Amazon S3 . . , Aurora Amazon S3 , .
• bucket-name - Amazon S3, . , .
• file-name-or-prefix - Amazon S3 XML , XML . , .

Answer 5

AmazonS3ReadOnlyAccess AmazonS3FullAccess , IAM. .

IAM → AWS, , , " ", S3 .

Answer 6

Amazon Aurora, , , . , .

Answer 7

, RDS S3. , .

Answer 8

.

1. sql 'LOAD' ( 220 ), -. , , RDS , S3. .

2. The error was reset immediately (0.2 s). Previously, I successfully downloaded data from S3, but all of a sudden, with the S3 URL changing, this error occurred again. I used the wrong S3 URL. Because I wanted to use the S3 prefix instead of the file. check the syntax "Download" to make your SQL correct.