All geek questions in one place

Data Sync from Amazon Aurora to Redshift

I am trying to set up synchronization between AWS Aurora and Redshift. What is the best way to achieve this synchronization?

Possible synchronization methods include: -

  • Query table to search for changes in the table (since I only do insertions, updates do not matter), export these changes to a flat file in the S3 bucket and use the Redshift copy command to paste into Redshift.

  • Use the python and Boto3 publisher to post changes to the Kinesis stream and then consume that stream in Firehose, from where I can copy directly to Redshift.

  • Use Kinesis Agent to detect changes to binlog (is it possible to detect int binlog changes using Kinesis Agent) and publish it to Firehose and copy to Firehose from there.

I have not studied Datapipeline AWS yet.

+2
source share
2 answers

As pointed out by @Mark B, AWS Data Migration can transfer data between databases. This can be done as a one-time exercise, or it can work continuously, keeping two databases in sync.

, Amazon Aurora Amazon Redshift .

+2

Source: https://habr.com/ru/post/1679756/

More articles:


All Articles