Where is the python descriptor state located?

Question

Background

I try to figure out Python descriptors by reading Lutz Learning Python in a section in which he says: "As properties, descriptors are designed to handle certain attributes ... Unlike properties, descriptors have their own state ..."

Throughout the chapter, he shows examples in which the managed attribute is actually hidden on the containing / wrapping object, as in:

def __set__(self, instance, value): instance._name = value.lower() 

I understand these examples, and they seem to be common in write up on this topic. However, their superiority over properties is not obvious to me, and they do not seem to correspond to the internal state promised in the above quote.

At the end of the chapter, he shows an example that is closer to what I pictured after reading “to have my own state”, as in:

 def __set__(self, instance, value): self.name = value.lower() 

The example runs, but does not do what I expect from it. Since the example is a bit long, I put it on Pastebin and added the last line that shows unexpected behavior (Bob's name is now Sue). Here's a shorter snippet of the demo:

 class Wrapper(object): class ExampleDescriptor(object): def __get__(self, instance, owner): print "get %s" % self.state return self.state def __set__(self, instance, value): print "set %s" % value self.state = value ex = ExampleDescriptor() w1 = Wrapper() w1.ex = 1 print w1.ex w2 = Wrapper() print w2.ex w2.ex = 2 print w1.ex print w1.ex is w2.ex 

The output of which:

 set 1 get 1 1 get 1 1 set 2 get 2 2 get 2 get 2 True 

None of this execution comes as a surprise after carefully examining the code. The validation logic in the descriptor makes de facto singleton of this attribute in the wrapper class; however, it is difficult to imagine that this divided state is Lutz’s intention or intention in this widely related textbook .

Question

Is it possible to create a descriptor that has an internal state unique to the wrapper object without saving this state on instances of the wrapper object (as in the first fragment)? Is it possible to change the CardHolder class from a related example so that Bob doesn't end as Sue?

Answer 1

"Like properties, descriptors are designed to handle certain attributes ... Unlike properties, descriptors have their own state ..."

I'm not sure what the Lutz point is trying to do, because the properties are, in fact, the descriptors themselves.

But, despite the fact that descriptors have their own state, this is not very useful, since, as you discovered, you get only one descriptor object for a class attribute instead of a single instance. This is why the instance is transferred, so unique unique values ​​can be stored / accessed.

To prove that this is one descriptor object for each attribute, you can try this slightly modified code with one of your links:

 class RevealAccess(object): """A data descriptor that sets and returns values normally and prints a message logging their access. """ def __init__(self, initval=None, name='var'): self.val = initval self.name = name def __get__(self, obj, objtype): print 'Retrieving', self.name return self.val def __set__(self, obj, val): print 'Updating' , self.name self.val = val class MyClass(object): x = RevealAccess(10, 'var "x"') y = RevealAccess(5, 'var "y"') m = MyClass() mx mx = 20 mx my 

What you will see:

 Retrieving var "x" Updating var "x" Retrieving var "x" Retrieving var "y" 

To answer your question: Yes. But it is a pain.

 class Stored(object): """A data descriptor that stores instance values in itself. """ instances = dict() def __init__(self, val): self.instances[self, None] = val def __get__(self, obj, objtype): return self.instances[self, obj] def __set__(self, obj, val): self.instances[self, obj] = val class MyClass(object): x = Stored(3) y = Stored(9) print(MyClass.x) print(MyClass.y) m = MyClass() mx = 42 print(mx) my = 19 print(my) print(mx) 

Answer 2

As already mentioned, the descriptor is an instance of the class, so its state is shared between each instance of the class.

The handle can store the internal hash of the instances it wraps. To avoid circular references, it would be wiser to have an instance id key. The only reason I would like to do this is to designate a descriptor to aggregate these properties from different instances.

As for the second part of your question, just follow, as you said, and save the base state of the instance instead of the handle, and then Bob will not be Sue.

Answer 3

(addition to other answers) To attach state to instances that you don't control without breaking them, just use a weak container; weakref.WeakKeyDictionary is appropriate here. The garbage collector will ensure that the descriptor’s additional state does not linger after the collection of instances, and that the descriptor does not cause the instances to live longer than usual.