C # Hand Carving

Question

Does anyone have any good resources that show creating an unlimited number of threads in C # without using ThreadPool?

I understand that you can ask the architecture of a system with hundreds or thousands of threads, so let me explain the task in case the CPU / OS makes this effort controversial.

I have about 2500 URLs that I need to check. Some of them are very slow: more than 10 seconds to answer. In any case, network latency is about 99.99% of each operation.

I would like to test all 2500 URLs as quickly as possible.

I hooked up a test that checks each of them in its thread.

The problem is that I'm using ThreadPool, and I think the default limit is 25, so no good. I need to manage them manually. Am I here to have lunch here?

I understand that CPU / OS is also likely to limit the number of parallel threads per processor, but I am sure this limit will be WAY above 25.

As for the architecture, I understand that I can block the entire cell if I need to connect 2 thousand HTTP streams, but this is an administrator task that runs in isolation and can take as many resources as available.

Thanks for your ideas.

Answer 1

Unable to create an unlimited number of threads. If you try, you will have many problems.

ThreadPool #. ThreadPool.SetMaxThreads, . , , , ( ).

Answer 2

, Windows XP (, , Vista/Win7) TCP-, (10). , , , .

Answer 3

, ThreadPool 256, , , . (Edit: Whoops - )

, :

Thread newThread = new Thread(new ThreadStart(myWorkerMethod));
newThread.Start();

, , , . , , .

Answer 4

ThreadPool; MiscUtil.Threading.CustomThreadPool MiscUtil ( Jon Skeet) , , , / .

; , , 50-1000 , , , , .

, HttpWebRequest HttpWebResponse URL; , , : ServicePointManager.DefaultConnectionLimit. WebRequests, (2 10), , .

Answer 5

, , 2/3 .

. , , . .

.

, , HTTP- .net , , .

, , , .

Answer 6

, ... , .

, ,

a.) MaxThreads 100. b.)

<system.net>
    <connectionManagement>
      <add address="*" maxconnection="100" />
    </connectionManagement>
  </system.net>

config

c.) TCP/IP XP 100 10.

d.) ServicePointManager.DefaultConnectionLimit 100.

.

, .

... , WebClient.DownloadStringAsync, , , .

-, WebClient/.NET, ...

Answer 7

2500 URL- , 2500 , , 2500. ThreadPool , -, .

, . , , . - .

Answer 8

, URL-, , , . , . , .

, , , , . , , ; 25 , . , .

( - ) app.config, maxconnection: http://msdn.microsoft.com/en-us/library/aa903351%28VS.71%29.aspx.

, "" , . , .

Answer 9

, C10K, ; , , Windows. # codeguru. IO, , , , , , . Windows TCP , .

Answer 10

You can run several applications of the application and assign it a higher priority in the task manager