MongoDB for DynamoDB

Question

I have a database currently in Mongo running on an EC2 instance, and you want to transfer the data to DynamoDB. Is this possible, and what is the most cost-effective way to achieve this?

Answer 1

When you ask for a “cost-effective way” to transfer data, I assume that you are looking for existing technologies that can make your life easier. If so, you can do the following:

• Export MongoDB data to a text file, say, in tsv format using mongoexport .
• Download this file somewhere in S3.
• Import this data into S3 in DynamoDB using AWS Data Pipeline .

Of course, you must develop and complete your DynamoDB table schema before doing all this.

Answer 2

Whenever you change databases, you have to be very careful how you transfer data. Some data formats support type consistency, while others do not.

Then there are only data formats that cannot process your circuit. For example, CSV does a great job of processing data when it is one row for each record, but how do you imagine the built-in array in CSV? It really is not possible, JSON is good at it, but JSON has its problems.

The simplest example is JSON and DateTime. JSON does not have a specification for storing DateTime values, they can end as ISO8601 dates, or perhaps UNIX Epoch timestamps, or really anything that a developer can think of. What about Longs, Doubles, Ints? JSON does not distinguish, it does all the lines, which can lead to a loss of precision if not deserialized correctly.

This makes it very important that you choose the right translator. Generally, you should minimize your own decision. This means loading drivers for both databases, reading a record from one, translating and writing to another. This is the best way to be absolutely sure that errors are handled properly for your environment, types are stored sequentially, and that the code correctly transfers the circuit from the source to the destination (if necessary).

What does all this mean to you? This means a lot of work for you. Someone may already have rolled something wide enough for your business, but I have already found in the past that it is best for you to do it yourself.

Answer 3

I know this post is old, Amazon did this using AWS DMS, check this document:

https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MongoDB.html 

Some relevant parts:

Using Amazon DynamoDB Database as a Target for AWS Database Migration Service

You can use AWS DMS to transfer data to an Amazon DynamoDB table. Amazon DynamoDB is a fully managed NoSQL database service that delivers fast and predictable performance with seamless scalability. AWS DMS supports using a relational database or MongoDB as a source.