Django model: dynamically change db_table

Question

I have a large number of data sets. Each dataset contains several database tables. The schema for the database table sets is identical. Each table set can have more than a million rows. Each data set refers to one task; there are no relations between tasks. One or more tasks belong to another user. Table sets are imported and eventually deleted as a table set. In terms of performance, it's best to keep them as separate sets of tables.

So, I would like to have several common Django models, one for each of several tables. I achieved this in the views.py file with code like this:

from foobar.models import Foo, Bar def my_view(request): prefix = request.GET.get('prefix') Foo._meta.db_table = prefix + '_foo' Bar._meta.db_table = prefix + '_bar' .... foobar_list = Foo.objects.filter(bar_id=myval) ... 

My questions are: is it safe to use this code with multiple users of a Django-based web application? Are model objects common to users? What happens if there are two requests at the same time?

EDIT No. 2: I reviewed Lee Ryan's answer and comments and came up with this code:

 from django.http import HttpResponse, HttpResponseNotFound from django.db import models from django.template import RequestContext, loader def getModels(prefix): table_map = {} table_map["foo"] = type(str(prefix + '_foo'), (models.Model,), { '__module__': 'foobar.models', 'id' : models.IntegerField(primary_key=True), 'foo' : models.TextField(blank=True), }) table_map["foo"]._meta.db_table = prefix + '_foo' table_map["bar"] = type(str(prefix + '_bar'), (models.Model,), { '__module__': 'foobar.models', 'id' : models.IntegerField(primary_key=True), 'foo' : models.ForeignKey(prefix + '_foo', null=True, blank=True), }) table_map["bar"]._meta.db_table = prefix + '_bar' return table_map def foobar_view(request): prefix = request.GET.get('prefix') if prefix != None and prefix.isdigit(): table_map = getModels(prefix) foobar_list = table_map["bar"].objects.filter.order_by('foo__foo') template = loader.get_template('foobar/foobar.html') context = RequestContext(request, { 'foobar_list': foobar_list, }) return HttpResponse(template.render(context)) else: return HttpResponseNotFound('<h1>Page not found</h1>') 

Now my question is, is this second draft editable code safe with multiple users at the same time?

Answer 1

This method is called sharding. No, this is not safe if you are doing parallel requests with threads.

What you can do is dynamically build multiple classes that point to different db_tables, and use the factory to select the class you want.

 tables = ["foo", "bar"] table_map = {} for tbl in tables: class T(models.Model): db_table = tbl ... table definition ... table_map[tbl] = T 

And then create a function that selects the correct table_map based on how you outline your data.

Also be careful with the injection if you take the table name from user input.

Alternatively, some database systems, such as PostgrSQL, allow you to use multiple schemas for each database, which may be the best way to separate your data in certain circumstances.