AWS SES S3 processes incoming email

Question

I am working on email publishing based on AWS SES. For all incoming emails, I set up routing to save messages to the S3 bucket so that I can process them asynchronously. The problem is that messages are saved in the S3 bucket in a raw format: headers, email body, etc. + Encrypted attachment (huge string) - all in one file.

Is there a way to split the email message separately from the attachment and save it as separate files at the AWS SES level? I am trying to get the data in the format that I need directly from AWS, and do not add another processing step to the process.

If AWS SES does not provide such a function, what would be the proper way to process these messages to get the result described above?

Answer 1

It is not possible for SES to automatically split your email. According to the documentation here :

Amazon SES provides you with raw, unmodified email, which is usually in the format of Multipurpose Email Extensions (MIME).

I would use S3 or SNS to run the Lambda function when SES puts the new email file in S3. The Lambda function can split the file as you like, and then write these new files to another S3 bucket.

Answer 2

For those who return later to this question, this is a reference to the JSON structure that you get when you call the Lambda function from SES.

http://docs.aws.amazon.com/ses/latest/DeveloperGuide/receiving-email-notifications-examples.html

This page took several searches; -)

From the link, the Lambda notice will look like this:

{ "notificationType": "Received", "receipt": { "timestamp": "2015-09-11T20:32:33.936Z", "processingTimeMillis": 406, "recipients": [ " [e-mail] " ], "spamVerdict": { "status": "PASS" }, "virusVerdict": { "status": "PASS" }, "spfVerdict": { "status": "PASS" }, "dkimVerdict": { "status": "PASS" }, "action": { "type": "S3", "topicArn": "arn:aws:sns:us-east-1:012345678912:example-topic", "bucketName": "my-S3-bucket", "objectKey": "\email" } }, "mail": { "timestamp": "2015-09-11T20:32:33.936Z", "source": " 0000014fbe1c09cf-7cb9f704-7531-4e53-89a1-5fa9744f5eb6-000000@ama zonses.com", "messageId": "d6iitobk75ur44p8kdnnp7g2n800", "destination": [ " [e-mail] " ], "headersTruncated": false, "headers": [ { "name": "Return-Path", "value": "< 0000014fbe1c09cf-7cb9f704-7531-4e53-89a1-5fa9744f5eb6-000000@ama zonses.com>" }, { "name": "Received", "value": "from a9-183.smtp-out.amazonses.com (a9-183.smtp-out.amazonses.com [54.240.9.183]) by inbound-smtp.us-east-1.amazonaws.com with SMTP id d6iitobk75ur44p8kdnnp7g2n800 for [e-mail] ; Fri, 11 Sep 2015 20:32:33 +0000 (UTC)" }, { "name": "DKIM-Signature", "value": "v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug; d=amazonses.com; t=1442003552; h=From:To:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Date:Message-ID:Feedback-ID; bh=DWr3IOmYWoXCA9ARqGC/UaODfghffiwFNRIb2Mckyt4=; b=p4ukUDSFqhqiub+zPR0DW1kp7oJZakrzupr6LBe6sUuvqpBkig56UzUwc29rFbJF hlX3Ov7DeYVNoN38stqwsF8ivcajXpQsXRC1cW9z8x875J041rClAjV7EGbLmudVpPX 4hHst1XPyX5wmgdHIhmUuh8oZKpVqGi6bHGzzf7g=" }, { "name": "From", "value": " [e-mail] " }, { "name": "To", "value": " [e-mail] " }, { "name": "Subject", "value": "Example subject" }, { "name": "MIME-Version", "value": "1.0" }, { "name": "Content-Type", "value": "text/plain; charset=UTF-8" }, { "name": "Content-Transfer-Encoding", "value": "7bit" }, { "name": "Date", "value": "Fri, 11 Sep 2015 20:32:32 +0000" }, { "name": "Message-ID", "value": "< [e-mail] >" }, { "name": "X-SES-Outgoing", "value": "2015.09.11-54.240.9.183" }, { "name": "Feedback-ID", "value": "1.us-east-1.Krv2FKpFdWV+KUYw3Qd6wcpPJ4Sv/pOPpEPSHn2u2o4=:AmazonSES" } ], "commonHeaders": { "returnPath": " 0000014fbe1c09cf-7cb9f704-7531-4e53-89a1-5fa9744f5eb6-000000@ama zonses.com", "from": [ " [e-mail] " ], "date": "Fri, 11 Sep 2015 20:32:32 +0000", "to": [ " [e-mail] " ], "messageId": "< [e-mail] >", "subject": "Example subject" } } } 

Answer 3

Regarding the question of how to write lambda. Here is a part of our Lambda. The main thing is to extract the parseEvent function from it. and data.event.Records [0], which will give you detailed information

 exports.handler = function(event, context, callback) { var AWS = require('aws-sdk'); // Validate characteristics of a SES event record. if (!event || !event.hasOwnProperty('Records') || event.Records.length !== 1 || event.Records[0].hasOwnProperty('eventSource') || event.Records[0].eventSource !== 'aws:ses' || event.Records[0].eventVersion !== '1.0') { callback(null, {'disposition':'STOP_RULE_SET'}); } email = data.event.Records[0].ses.mail; subjectLine = event.Records[0].ses.mail.commonHeaders.subject; } 

The key is event.Record [0] .ses.mail. Unfortunately, I can’t find its structure using Google search, I'm sure I saw it before.