Azure / AWS ORM Design Guide

Question

I develop and implement .Net ORM, which should support both Azure Storage (tables, queues, blobs) and AWS Storage (EBS, SimpleDB, S3) and hide all implementation details behind a common interface. The main goal of the project is simplicity.

Part of the work was done at http://www.cs.virginia.edu/~humphrey/papers/CSAL.pdf , but their proposed interface, in my opinion, is too closely related to Azure / AWS and is likely to be broken if new features will be added or old ones changed. For example, I don’t care that I can create / delete tables, I just need to save an object of a certain type in the most efficient way.

So, I would like to ask you to share your experience on this issue in the form of recommendations (DO, CONSIDER, AVOID, DO NOT). I would really appreciate an understanding starting with the general principles of ORM design and ending with the exact level of abstraction, which is likely to continue taking into account the most likely evolution paths of Azure and AWS.

Answer 1

If you really want to avoid hard communication, I suggest that you simply implement and abstract the layer on top of the minimum set of functions available on both types of storages.

But in any case, what you are trying to do does not make sense to me. The cloud (non-relational) storage is so peculiar that an attempt to create a common ORM will be a gigantic mistake. Modern SQL ORMs are “good” because RDBMS have a common minimum set of functions, which is actually quite extensive, and in most cases you don't need exotic things. With cloud storage, each implementation is almost unique, precisely because the very first aspect to keep in mind is scalability. For example, if you throw away Blob in Azure because they don’t exist in Amazon (I have no idea, I just make an example), then it will probably be difficult for you to control the simultaneous blob access in Amazon and in Lazur too, and it doesn’t have meaning.

I would prefer to implement a data access layer that is reasonably designed to avoid problems, but use ALL the available features on both platforms (with very different implementations, if necessary).