Is it possible to use the same CookieContainer through multiple HttpWebRequests?

Question

I am doing some kind of WebCrawler, and I need to keep the state of Cookies between requests.

I load all async pages, creating new instances of HttpWebRequest, but setting the same CookieContainer. Pages can write and read cookies.

Can I do it safely? Is there any alternative that is not a subclass of CookieContainer that puts locks on all methods?

MSDN says this class is not thread safe, but in practice can I do this?

Answer 1

According to the documentation :

All public static (Shared in Visual Basic) members of this type are safe thread. Any instance members are not guaranteed to be safe threads.

So, you must ensure the correct lock if you want to share the same instance between multiple threads. But since the members of the CookieContainer class CookieContainer not actually processed by your code, but implicitly from the different HttpWebRequest instances that you created, it was not easy to synchronize them correctly, except, of course, blocking your requests, which, of course, are defeated by the goal and the level of parallelism, which I suppose you are trying to reach here.

In practice, you will have problems, this is another topic. The fact is that the documentation (and therefore the author) does not provide you with any guarantees.

Answer 2

Actually, if you look at the source code of CookieContainer here . Despite the documentation, it seems to be thread safe.

Good answer that explains this.

fooobar.com/questions/1403954 / ...

You will notice that the author of CookieContainer made sure to use lock {} and SyncRoot around these parts, which change the code collection, and I do not think that this approach is not addressed for parallel scripts.

So, as a general rule

When you see the standard documentation Any instance members are not guaranteed to be thread safe. , do not stop there and do not lock yourself. Use reflection or sourcesource.microsoft.com (for .NET classes) to find out if they are thread safe or not.