Python circular import again (aka wrong with this design)

Question

Consider the python (3.x) scripts:

main.py:

from test.team import team from test.user import user if __name__ == '__main__': u = user() t = team() u.setTeam(t) t.setLeader(u) 

Test / user.py:

 from test.team import team class user: def setTeam(self, t): if issubclass(t, team.__class__): self.team = t 

Test / team.py:

 from test.user import user class team: def setLeader(self, u): if issubclass(u, user.__class__): self.leader = u 

Now, of course, I have circular imports and a great ImportError.

So, not being pythonista, I have three questions. Primarily:

I am. How can I make this work work?

And, knowing that someone will inevitably say: “Circular imports always indicate a design problem,” the second question arises:

II. Why is this design bad?

And finally, the third:

III. What would be a better alternative?

To be precise, type checking, as mentioned above, is just an example; there is also an index layer based on the class, which allows, for example. find all users that are members of one team (the user class has many subclasses, so the index doubles for users in general and for each particular subclass) or all the teams that provided the user as a member

Edit:

Hope a more detailed example clarifies what I'm trying to achieve. Files omitted for reading (but having one source file of 300 KB scares me somehow, so please assume that each class is in a different file)

 # ENTITY class Entity: _id = None _defs = {} _data = None def __init__(self, **kwargs): self._id = uuid.uuid4() # for example. or randint(). or x+1. self._data = {}.update(kwargs) def __settattr__(self, name, value): if name in self._defs: if issubclass(value.__class__, self._defs[name]): self._data[name] = value # more stuff goes here, specially indexing dependencies, so we can # do Index(some_class, name_of_property, some.object) to find all # objects of some_class or its children where # given property == some.object else: raise Exception('Some misleading message') else: self.__dict__[name] = value def __gettattr__(self, name): return self._data[name] # USERS class User(Entity): _defs = {'team':Team} class DPLUser(User): _defs = {'team':DPLTeam} class PythonUser(DPLUser) pass class PerlUser(DPLUser) pass class FunctionalUser(User): _defs = {'team':FunctionalTeam} class HaskellUser(FunctionalUser) pass class ErlangUser(FunctionalUser) pass # TEAMS class Team(Entity): _defs = {'leader':User} class DPLTeam(Team): _defs = {'leader':DPLUser} class FunctionalTeam(Team): _defs = {'leader':FunctionalUser} 

and now some use:

 t1 = FunctionalTeam() t2 = DLPTeam() t3 = Team() u1 = HaskellUser() u2 = PythonUser() t1.leader = u1 # ok t2.leader = u2 # ok t1.leader = u2 # not ok, exception t3.leader = u2 # ok # now , index print(Index(FunctionalTeam, 'leader', u2)) # -> [t2] print(Index(Team, 'leader', u2)) # -> [t2,t3] 

So, it works great (implementation details are omitted, but nothing complicated) besides this wicked circular import thing.

Answer 1

Circular imports are not inherently bad. It is natural for team code to rely on user , and user does something with team .

Worse practice here is from module import member . The team module tries to get the user class during import, and the user module tries to get the team class. But the team class does not exist yet because you are still in the first line of team.py when user.py is user.py .

Instead, import only modules. This leads to a clearer namespace, makes possible the subsequent patch of the monkey, and solves the import problem. Since you import a module only during import, you don’t care what the class inside it is not yet defined. By the time you start using the class, it will be.

So test / users.py:

 import test.teams class User: def setTeam(self, t): if isinstance(t, test.teams.Team): self.team = t 

Test / teams.py:

 import test.users class Team: def setLeader(self, u): if isinstance(u, test.users.User): self.leader = u 

from test import teams and then teams.Team is OK too if you want to write test less. It still imports the module, not a member of the module.

In addition, if team and user relatively simple, put them in the same module. You do not need to follow the Java one-class-per-file idiom. isinstance testing and set methods also scream to me a dissolute-Java-wart; depending on what you are doing, you might well be better off using a simple, untested @property type.

Answer 2

I am. To make it work, you can use deferred import. One way is to leave user.py on your own and change the team.py command to:

 class team: def setLeader(self, u): from test.user import user if issubclass(u, user.__class__): self.leader = u 

III. For an alternative, why not put the commands and user classes in the same file?

Answer 3

Bad practice / smelly are the following things:

• A sample of unnecessary type checking ( see also here ). Just use the objects you get as a user / team and throw an exception (or, in most cases, one raises without the need for additional code) when it breaks. Leave this and you circular import leave (at least for now). As long as the objects you receive behave like a user / team, they can be anything. ( Duck Typing )
• lowercase classes (this is more or less a matter of taste, but the generally accepted standard ( PEP 8 ) does it differently
• setter where not needed: you could say: my_team.leader=user_b and user_b.team=my_team
• data consistency problems: what if (my_team.leader.team!=my_team) ?

Answer 4

Here is what I have not seen. Is this a bad idea / design using sys.modules directly? After reading the @bobince solution, I thought I understood the whole import business, but then I ran into a problem similar to the question that refers to this one.

Here is another solution:

 # main.py from test import team from test import user if __name__ == '__main__': u = user.User() t = team.Team() u.setTeam(t) t.setLeader(u) 

---

 # test/team.py from test import user class Team: def setLeader(self, u): if isinstance(u, user.User): self.leader = u 

---

 # test/user.py import sys team = sys.modules['test.team'] class User: def setTeam(self, t): if isinstance(t, team.Team): self.team = t 

and the file test/__init__.py empty. The reason for this is that test.team imported test.team . When python imports / reads a file, it adds a module to sys.modules . When we import test/user.py , the test/user.py module test.team already be defined, since we import it into main.py

I like this idea for modules that grow quite large, but there are functions and classes that depend on each other. Suppose there is a file called util.py , and this file contains many classes that depend on each other. Perhaps we could split the code between different files that depend on each other. How do we get around round imports?

Well, in the util.py file util.py we just import all the objects from other "private" files, I say private, since these files are not intended for direct access, instead we access them through the source file:

 # mymodule/util.py from mymodule.private_util1 import Class1 from mymodule.private_util2 import Class2 from mymodule.private_util3 import Class3 

Then on each of the other files:

 # mymodule/private_util1.py import sys util = sys.modules['mymodule.util'] class Class1(object): # code using other classes: util.Class2, util.Class3, etc 

---

 # mymodule/private_util2.py import sys util = sys.modules['mymodule.util'] class Class2(object): # code using other classes: util.Class1, util.Class3, etc 

The sys.modules call will work until mymodule.util is imported mymodule.util .

Finally, I will only point out that this is done in order to help users with readability (shorter files), and therefore I would not say that circular imports are “inherently” bad. Everything could be done in the same file, but we use it so that we can separate the code and not confuse ourselves while scrolling through a huge file.