Python unpacks an object with an instance of the class inside

Question

I am using python query library and trying to save a session.

Since I have multiple IP addresses on my host, I created the following method to associate a session with a specific IP address.

class SourceAddressAdapter(HTTPAdapter): def __init__(self, source_address, **kwargs): self.source_address = source_address super(SourceAddressAdapter, self).__init__(**kwargs) def init_poolmanager(self, connections, maxsize, block=False): self.poolmanager = PoolManager(num_pools=connections, maxsize=maxsize, block=block, source_address=self.source_address) 

The following code fragment is used to call this class:

 r = requests.Session() r.mount('http://', SourceAddressAdapter((self.ip,0))) r.mount('https://', SourceAddressAdapter((self.ip,0))) 

After installing the http and https protocol on this adapter, I used pickle to save the object in redis as follows:

 session = pickle.dumps(r) redis.hset('sessions',id,session) 

The problem arose when I tried to expand the session object:

 s=redis.hget('sessions', id) pickle.loads(s) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python2.7/pickle.py", line 1382, in loads return Unpickler(file).load() File "/usr/lib/python2.7/pickle.py", line 858, in load dispatch[key](self) File "/usr/lib/python2.7/pickle.py", line 1217, in load_build setstate(state) File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 114, in __setstate__ block=self._pool_block) File "network_driver.py", line 158, in init_poolmanager source_address=self.source_address) AttributeError: 'SourceAddressAdapter' object has no attribute 'source_address' 

He complained that the SourceAddressAdapter did not have a source_address attribute. Before I included this SourceAddressAdapter class in my session, serialization worked fine.

So, I think this is a problem with setting up class trace / markup.

---

UPDATE:

It works after I added the __getstate__ and __setstate__ to the SourceAddressAdapter

 def __getstate__(self): # it calls HTTPAdapter __getstate__() state = super(SourceAddressAdapter, self).__getstate__() state['source_address'] = self.source_address return state def __setstate__(self,state): self.source_address = state['source_address'] # Call HTTPAdapter __setstate__ function to pack the attributes in parent class super(SourceAddressAdapter, self).__setstate__(state) 

Answer 1

I believe the problem is that the HTTPAdapter class defines the __setstate__ method. This function is called unpickling and restores an instance in a marinated state. However, the HTTPAdapter does not know anything about your source_address attribute, so the attribute is not restored (or perhaps not even pickled).

To fix this, you need to override the __setstate__ function, something like this:

 def __setstate__(self, state): self.source_address= state['source_address'] # do this before calling __setstate__ HTTPAdapter.__setstate__(self, state) 

And, as mentioned earlier, you may also need to override the __getstate__ function __getstate__ that source_address becomes pickled.