Adding a “trigger” load after data

Question

We have data APIs that feed data to our data warehouse in real time. Until now, we have excluded the field with PII, however, we would like to start to incorporate emailand nameas part of the feed.

Please note that after adding these two fields, we can only use emailand nameif the other field with the name is partnernot null (this is not something that we can configure through the API, so do it after loading the data) due to contractual obligations.

So, if partnerit is null: emailand nameshould be overwritten as null. If partnernot null: emailand nameshould remain as they are (provided by the API).

Could you help in setting up the appropriate trigger for this Redshift table? i.e. let the API load as is, and after each load of the line - if partnerit is null - make emailand addressnull. Thank!

Answer 1

Unfortunately, there is no trigger system in redshift.

If you cannot do this conversion on the send side or inside the stitch, and it is important that the data is safe, this is what I would do.

• Set up a line to write data to the redshift stage circuitry, which has blocked security so that ordinary users cannot see the data.
• redshift etl ( crontab, ) "" "" . . sql "".

: postgres , postgres, AWS DMS . , .