Python pickle crash when trying to return default to __getattr__

Question

I have a dictionary like a class that I use to store some values ​​as attributes. I recently added some logic ( __getattr__ ) to return None if the attribute does not exist. As soon as I made this pickle, it crashed, and I wanted to understand why?

Test code:

 import cPickle class DictionaryLike(object): def __init__(self, **kwargs): self.__dict__.update(kwargs) def __iter__(self): return iter(self.__dict__) def __getitem__(self, key): if(self.__dict__.has_key(key)): return self.__dict__[key] else: return None ''' This is the culprit...''' def __getattr__(self, key): print 'Retreiving Value ' , key return self.__getitem__(key) class SomeClass(object): def __init__(self, kwargs={}): self.args = DictionaryLike(**kwargs) someClass = SomeClass() content = cPickle.dumps(someClass,-1) print content 

---

Result:

 Retreiving Value __getnewargs__ Traceback (most recent call last): File <<file>> line 29, in <module> content = cPickle.dumps(someClass,-1) TypeError: 'NoneType' object is not callable` 

---

Did I do something stupid? I read a post that deepcopy () might require me to throw an exception if the key does not exist? If so, is there an easy way to achieve what I want without throwing an exception?

End result: if some challenges

 someClass.args.i_dont_exist 

I want him to return None.

Answer 1

Implementing __getattr__ bit complicated, as it is called for every non-existent attribute. In your case, the pickle module checks your class for the special __getnewargs__ method and gets None , which obviously cannot be called.

You might want to modify __getattr__ to invoke the base implementation for magic names:

 def __getattr__(self, key): if key.startswith('__') and key.endswith('__'): return super(DictionaryLike, self).__getattr__(key) return self.__getitem__(key) 

I usually pass all names, starting with underscores, so that I can bypass the magic for internal characters.

Answer 2

You need to raise an AttributeError if the attribute is not in your class:

 def __getattr__(self, key): i = self.__getitem__(key) if i == None: raise AttributeError return self.__getitem__(key) 

I am going to suggest that this behavior is required. From the python documentation for getattr : “Called when the attribute search did not find the attribute in normal places (i.e., it is not an instance of the attribute and is not found in the class tree for itself). Name is the attribute name. This method should return (calculated) attribute value or raise an AttributeError exception. "

It is impossible to tell pickle etc. that the attribute it is looking for is not found unless you throw an exception. For example, in an error message, pickle searches for a special method called __getnewargs__ , pickle expects that if an AttributeError exception is not found, the return value will be called.

I assume that one potential work around you might perhaps try to identify all the special methods that pickled are looking for as fictitious methods?