Running the COPY command to load gzip-ed data into Redshift in S3

When I ran the copy command to copy all the files from the S3 folder to the Redshift table, it does not work with : "ERROR: gzip: unexpected end of stream. Unknown zlib error code. zlib error code: -1"

copy table_name 
    (column_list)
from 's3://bucket_name/folder_name/'
     credentials 'aws_access_key_id=xxxxxx;aws_secret_access_key=xxxxxxxxx'
     delimiter '|' GZIP

However, when I specify the file prefix for each of the files in the folder, it succeeds:

copy table_name 
    (column_list)
from 's3://bucket_name/folder_name/file_prefix'
     credentials 'aws_access_key_id=xxxxxx;aws_secret_access_key=xxxxxxxxx'
     delimiter '|' GZIP

Files have gzip-ed.

The AWS document does not explicitly state that if you simply specify the folder_name, it will be fine if the copy command loads the entire contents of this folder, however I get an error message.

Has anyone encountered similar problems? Is a file prefix required for this kind of operation?

+4
source share
2 answers

gzipped . GZip "" , .

, , , , Redshift.

...; -)

+5

, gzip , , .

- "S3 Browser". , . , gzip .

+2

Source: https://habr.com/ru/post/1570857/


All Articles