.NET NetworkStream Read Slowness

Question

I have a network code to handle an arbitrary TCP connection.

Everything works as expected, but seems slow. When I was profiling the code, it seems to have spent a good 600 ms in NetworkStream.Read (), and I am wondering how to improve it. I was looking for buffer sizes and alternating between a massive buffer to read all the data in one go or a small one that should combine the data into a StringBuilder. The client I'm using is currently a web browser, but this code is generic and it may not be the HTTP data that is sent to it. Any ideas?

My code is:

public void StartListening() { try { lock (oSyncRoot) { oTCPListener = new TcpListener(oIPaddress, nPort); // fire up the server oTCPListener.Start(); // set listening bit bIsListening = true; } // Enter the listening loop. do { // Wait for connection TcpClient newClient = oTCPListener.AcceptTcpClient(); // queue a request to take care of the client oThreadPool.QueueUserWorkItem(new WaitCallback(ProcessConnection), newClient); } while (bIsListening); } catch (SocketException se) { Logger.Write(new TCPLogEntry("SocketException: " + se.ToString())); } finally { // shut it down StopListening(); } } private void ProcessConnection(object oClient) { TcpClient oTCPClient = (TcpClient)oClient; try { byte[] abBuffer = new byte[1024]; StringBuilder sbReceivedData = new StringBuilder(); using (NetworkStream oNetworkStream = oTCPClient.GetStream()) { // set initial read timeout to nInitialTimeoutMS to allow for connection oNetworkStream.ReadTimeout = nInitialTimeoutMS; int nBytesRead = 0; do { try { bool bDataAvailable = oNetworkStream.DataAvailable; while (!bDataAvailable) { Thread.Sleep(5); bDataAvailable = oNetworkStream.DataAvailable; } nBytesRead = oNetworkStream.Read(abBuffer, 0, abBuffer.Length); if (nBytesRead > 0) { // Translate data bytes to an ASCII string and append sbReceivedData.Append(Encoding.UTF8.GetString(abBuffer, 0, nBytesRead)); // decrease read timeout to nReadTimeoutMS second now that data is coming in oNetworkStream.ReadTimeout = nReadTimeoutMS; } } catch (IOException) { // read timed out, all data has been retrieved nBytesRead = 0; } } while (nBytesRead > 0); //send the data to the callback and get the response back byte[] abResponse = oClientHandlerDelegate(sbReceivedData.ToString(), oTCPClient); if (abResponse != null) { oNetworkStream.Write(abResponse, 0, abResponse.Length); oNetworkStream.Flush(); } } } catch (Exception e) { Logger.Write(new TCPLogEntry("Caught Exception " + e.StackTrace)); } finally { // stop talking to client if (oTCPClient != null) { oTCPClient.Close(); } } } 

Edit: I get roughly the same numbers on two completely different machines (my XP development machine and field 2003 in the ear). I put some time in the code around the relevant parts (using System.Diagnostic.StopWatch) and uploaded it to the log:

  6/6/2009 3:44:50 PM: Debug: While DataAvailable took 0 ms
 6/6/2009 3:44:50 PM: Debug: Read took 531 ms
 6/6/2009 3:44:50 PM: Debug: ProcessConnection took 577 ms

Answer 1

After a few more studies, it seems that the only way to speed it up is to break after the first x bytes have been read. The delay seems to be on the second reading. If I changed the buffer to 8096 bytes (maybe my maximum application will be sent any time) and break here:

  if (nBytesRead > 0) { // Translate data bytes to an ASCII string and append sbReceivedData.Append(Encoding.UTF8.GetString(abBuffer, 0, nBytesRead)); if (bTurboMode) { break; } else { // decrease read timeout to nReadTimeoutMS second now that data is coming in oNetworkStream.ReadTimeout = nReadTimeoutMS; } } 

Then the response time is from 600 ms to 80 ms. This is an acceptable solution for me at the moment. I can switch bTurboMode from the calling application and speed up the process in this case

Answer 2

I recommend you use Microsoft Network Monitor or something like that to find out what happens in terms of these 600 ms. NetworkStream is part of network software — when you look at its behavior, always consider what the network does.

Answer 3

Another vote for using network monitoring software. It should do either a network monitor or WireShark. Make sure you record what time the networkstream.read call begins and ends, and you can find out where your program events occurred in the recorded network traffic.

In addition, I would recommend that the NetworkStream.DataAvailable property become true before you call the Read method and record the time when it becomes true. If your network monitor shows data reaching 600 ms before your program indicates that it can be read, something else on your computer may support the package. antivirus or your firewall.

Adding 2009/7/6 3:12 PM EDT:

The additional information about the time you sent is interesting. If data is available, why has it been reading so long? I ran your code on my development machine, and both expected the data to be available, and the read function itself would exit in 0 milliseconds. Are you sure that the latest service packs, etc. are installed? I am running Visual Studio Professional 2005 with .NET 2.0.50727. I also have .NET 3.0 and 3.5 installed, but I don't think VS 2005 uses them. Do you have a new OS installation (real or virtual machine) without any additional programs (even including the “required” corporate IT) that you could try?

Here is the code I ran:

 using System; using System.Collections.Generic; using System.Text; using System.Net; using System.Net.Sockets; using System.IO; using System.Threading; using System.Diagnostics; namespace stackoverflowtest { class Program { static private object oSyncRoot = new object(); static private TcpListener oTCPListener; static private IPAddress oIPaddress = IPAddress.Parse("10.1.1.109"); static private int nPort = 8009; static bool bIsListening = true; static void Main(string[] args) { StartListening(); Thread.Sleep(500000); bIsListening = false; } public static void StartListening() { try { lock (oSyncRoot) { oTCPListener = new TcpListener(oIPaddress, nPort); // fire up the server oTCPListener.Start(); // set listening bit bIsListening = true; } // Enter the listening loop. do { // Wait for connection TcpClient newClient = oTCPListener.AcceptTcpClient(); // queue a request to take care of the client ThreadPool.QueueUserWorkItem(new WaitCallback(ProcessConnection), newClient); } while (bIsListening); } catch (SocketException se) { Console.WriteLine("SocketException: " + se.ToString()); } finally { // shut it down //StopListening(); } } private static void ProcessConnection(object oClient) { TcpClient oTCPClient = (TcpClient)oClient; try { byte[] abBuffer = new byte[1024]; StringBuilder sbReceivedData = new StringBuilder(); using (NetworkStream oNetworkStream = oTCPClient.GetStream()) { int nInitialTimeoutMS = 1000; // set initial read timeout to nInitialTimeoutMS to allow for connection oNetworkStream.ReadTimeout = nInitialTimeoutMS; int nBytesRead = 0; do { try { bool bDataAvailable = oNetworkStream.DataAvailable; Stopwatch sw = new Stopwatch(); while (!bDataAvailable) { Thread.Sleep(5); bDataAvailable = oNetworkStream.DataAvailable; } Console.WriteLine("DataAvailable loop took " + sw.ElapsedMilliseconds); sw.Reset(); nBytesRead = oNetworkStream.Read(abBuffer, 0, abBuffer.Length); Console.WriteLine("Reading " + nBytesRead + " took " + sw.ElapsedMilliseconds); if (nBytesRead > 0) { // Translate data bytes to an ASCII string and append sbReceivedData.Append(Encoding.UTF8.GetString(abBuffer, 0, nBytesRead)); // decrease read timeout to nReadTimeoutMS second now that data is coming in int nReadTimeoutMS = 100; oNetworkStream.ReadTimeout = nReadTimeoutMS; } } catch (IOException) { // read timed out, all data has been retrieved nBytesRead = 0; } } while (nBytesRead > 0); byte[] abResponse = new byte[1024]; for (int i = 0; i < abResponse.Length; i++) { abResponse[i] = (byte)i; } oNetworkStream.Write(abResponse, 0, abResponse.Length); oNetworkStream.Flush(); //send the data to the callback and get the response back //byte[] abResponse = oClientHandlerDelegate(sbReceivedData.ToString(), oTCPClient); //if (abResponse != null) //{ // oNetworkStream.Write(abResponse, 0, abResponse.Length); // oNetworkStream.Flush(); //} } } catch (Exception e) { Console.WriteLine("Caught Exception " + e.StackTrace); } finally { // stop talking to client if (oTCPClient != null) { oTCPClient.Close(); } } } } }