How and when to properly use weakref in Python

Question

I have a code in which class instances have a parent ↔ child link to each other, for example:

class Node(object): def __init__(self): self.parent = None self.children = {} def AddChild(self, name, child): child.parent = self self.children[name] = child def Run(): root, c1, c2 = Node(), Node(), Node() root.AddChild("first", c1) root.AddChild("second", c2) Run() 

I think this creates circular links, so that root , c1 and c2 will not be freed after Run () is complete, right ?. So how to free them? I think I can do something like root.children.clear() or self.parent = None - but what if I don't know when to do it?

Is this a good time to use weakref? What exactly, I poorly understand? parent attribute? children attribute The whole object? All of the above? I see talk about WeakKeyDictionary and weakref.proxy, but it's unclear how to use them, if at all possible.

This is also on python2.4 (cannot be updated).

Update: Example and Summary

Which objects for weakref-ify depend on which object can live without the other, and which objects depend on each other. The object that lives the longest should contain weak references to objects with a shorter period. Similarly, weaknesses should not be related to dependencies - if they are, the addiction can disappear silently, even if it is still needed.

If, for example, you have a tree structure, root , which has children, kids , but can exist without children, then the root object should use weakrefs for its kids . This also occurs if the child is dependent on the existence of the parent. Below, the child requires a parent to calculate its depth, and therefore strong-ref for parent . Elements of the kids attribute are optional, so weak links are used to prevent circular references.

 class Node: def __init__(self) self.parent = None self.kids = weakref.WeakValueDictionary() def GetDepth(self): root, depth = self, 0 while root: depth += 1 root = root.parent return depth root = Node() root.kids["one"] = Node() root.kids["two"] = Node() # do what you will with root or sub-trees of it. 

To flip the relationship around, we have something like below. Here, Facade classes require the Subsystem instance to work, so they use a strong reference to the subsystem they need. Subsystem s, however, does not require Facade . Subsystem simply specify how Facade notified of each other's actions.

 class Facade: def __init__(self, subsystem) self.subsystem = subsystem subsystem.Register(self) class Subsystem: def __init__(self): self.notify = [] def Register(self, who): self.notify.append(weakref.proxy(who)) sub = Subsystem() f1 = CliFacade(sub) f2 = WebFacade(sub) # Go on to reading from POST, stdin, etc 

Answer 1

Yes, the weak here is excellent. In particular, instead of:

 self.children = {} 

using:

 self.children = weakref.WeakValueDictionary() 

Nothing needs to change your code. Thus, when the child has no other differences, he simply leaves - and therefore the entry in the parent card is children , which has this child as a value.

Avoiding reference loops is highly appreciated using caches as the motivation for using the weakref module. Ref loops will not kill you, but they can end up clogging your memory, especially. if some of the classes in which the instances are involved define __del__ , as this prevents the gc module from dissolving these loops.

Answer 2

I suggest using child.parent = weakref.proxy(self) . This is a good solution to avoid circular references in the case where the lifetime (external links to) of the parent covers the lifetime of the child . Contrary, use weakref for child (as Alex suggested) when child 's lifetime spans parent lifetime. But never use weakref when both parent and child can be alive without the others.

Here, these rules are illustrated by examples. Use weakref-ed parent if you store root in some variable and pass it while the children are accessing them:

 def Run(): root, c1, c2 = Node(), Node(), Node() root.AddChild("first", c1) root.AddChild("second", c2) return root # Note that only root refers to c1 and c2 after return, # so this references should be strong 

Use weakref-ed children if you bind all of them to variables, and through them you can access through root:

 def Run(): root, c1, c2 = Node(), Node(), Node() root.AddChild("first", c1) root.AddChild("second", c2) return c1, c2 

But none of them will work for the following:

 def Run(): root, c1, c2 = Node(), Node(), Node() root.AddChild("first", c1) root.AddChild("second", c2) return c1 

Answer 3

I wanted to clarify which links might be weak. The following approach is general, but I use a doubly connected tree in all examples.

Logical step 1.

You need to make sure that there are compelling links to keep all the objects alive until you need them. This can be done in many ways, for example:

• [direct names]: a named link for each node in the tree
• [container]: link to the container that stores all the nodes
• [root + children]: a link to the root of the node and links from each node to its children
• [leaves + parent]: links to all leaf nodes and links from each node to the parent

Logical step 2.

Now you add links to present information, if necessary.

For example, if you used the [container] approach in step 1, you still have to represent the edges. An edge between nodes A and B can be represented with one link; he can go in any direction. Again, there are many options, for example:

• [children]: links from each node to its children
• [parent]: link from each node to the parent
• [set of sets]: a set containing 2-element sets; each 2-element contains links to nodes of one edge

Of course, if you used the [root + children] approach in step 1, all of your information is already fully presented, so you will skip this step.

Logical step 3.

You are now adding links to improve performance, if necessary.

For example, if you used the [container] approach in step 1 and [children] in step 2, you might need to improve the speed of certain algorithms and add links between each node and its parent. Such information is logically redundant because you can (at a cost in performance) extract it from existing data.

---

All links in step 1 should be strong.

All links in steps 2 and 3 may be weak or strong. It makes no sense to use strong links. There is the advantage of using weak references until you find out that loops are no longer possible. Strictly speaking, once you know that loops are impossible, it doesn’t matter whether to use weak or strong links. But in order not to think about it, you could use exceptionally weak links in steps 2 and 3.