I am starting to develop a few large enterprise platforms in Python and wonder if you guys can give me some tips on how to organize the various components and which packages will help me achieve the goals of scalability, maintainability and reliability.
A system is basically a service that collects data from various external sources, with each external source having its own separate application. These applications will poll the central database and receive any queries that were sent to an external source.
The main website and the REST / SOAP API will appear, which should also have access to the central data service.
My initial thought was to use Django for the website, web service and data access level (using the built-in ORM), and then external source applications can use web services to get the information they need to process the request and save results. Using this method will allow me to have multiple instances of service applications running on the same computers in order to balance the load. Are there any more elegant means to achieve this? I heard about messaging systems like MQ, can this be useful in this scenario?
My other thought was to use a completely separate data service not based on Django, and to use some kind of remote or remote objects (in their existence in Python) to interact with the data model. The downside here would be with a website that would become much slower if it had to push all its data requests through the second level.
I would really like to hear what other developers have come up with to achieve these goals in the most flexible way.
source
share