In your example, each contact can have one email address, and each email address can belong to several contacts. This is the wrong way, i.e. You must put the ForeignKey in the email model.
.
class Email(models.Model):
email = models.EmailField()
user = models.ForeignKey(User)
u = User.objects.get(pk=1)
u.email_set.all()