Named Pipe Performance Issues

Question

I use named pipes for inter-procedure communication between C # and Delphi. C # uses the System.IO.Pipes package, while Delphi uses Libby pipes.pas . Unfortunately, communication practically does not differ in high performance: Profiling showed me that the message occupies 72% of the entire execution environment, while the rest are used by calculations.
I was able to find one problem that could take resources. Unless I explicitly disconnect the connection to send the client to Delphi, C # does not receive any data at all.

Delphi (sending)

 FClient1.Write(msg[1], Length(msg)); FClient1.FlushPipeBuffers; FClient1.WaitForReply(20); FClient1.Disconnect; // disconnect to signalize C# that the writing is finished FClient1.Connect; // connect again to prevent synchronization problems 

C # (receive)

 // Wait for a client to connect stc.pipeServer.WaitForConnection(); while (reconnect_attempts < MAX_RECONNECT_ATTEMPTS) // { string tmp = sr.ReadLine(); // if result is empty, try again for <MAX_RECONNECT_ATTEMPTS> times // so you can eliminate the chance that there just a single empty request while (tmp != null)// && result != tmp) { tmp = sr.ReadLine(); result += tmp; } // sleep, increment reconnect, write debugging... } stc.pipeServer.Close(); 

Even if I assume reconnecting is expensive, I'm not quite sure about it. One data stream (approximately 1/11 kb) takes 130 (respectively 270 ms for 11kb) in total (sending and receiving).

My question would be :
Is it necessary to forcefully disconnect pipes to signal that the client is written? As for my observations, this is only necessary when sending using libby's. Are there other possible causes for poor performance? Thanks in advance.

As an addition, they are sent here and get the other way around:

C # (submit)

  stc.pipeClient.Connect(); StreamWriter sw = new StreamWriter(stc.pipeClient); //sw.AutoFlush = true; sw.WriteLine(msg); sw.Flush(); stc.pipeClient.WaitForPipeDrain(); // waits for the other end to read all bytes // neither disconnect nor dispose 

Delphi (receive)

  SetLength(S, Stream.Size); Stream.Read(S[1], Length(S)); FPipeBuffer := FPipeBuffer + S; { TODO 2 : switch case ID } // if the XML is complete, ie ends with the closing checksum if (IsFullMessage()) then begin // end reading, set flag FIsPipeReady := true; end 

Answer 1

After a large (manual) profiling, I came up with two ideas about the problem:

• Libby pipes are a complex beast. Since it seems to use multiple threads and show strange behavior regarding its use, manual use of WinApi was more convincing in the end. Moreover, the performance resulting from the actual message has increased. In other words: in a relatively simple IPC scenario like this, libby channels seem slower than WinApi.
• Anonymous pipes / using stdout and stdin seem even faster than named pipes.

However, I must add that I'm still a little confused and cannot say if this is really true, or I crunched the wrong numbers here.

Here is a simple example of what a WinApi implementation in Delphi might look like:

 // setup pipes, you'll need one for each direction // init handles with 0 CreatePipe(ReadPipe1, // hReadpipe WritePipe1, // hWritePIpe @SecurityAttributes, // Security PIPE_SIZE) // Size // setup Startupinfo FillChar(StartupInfo, Sizeof(StartupInfo), 0); StartupInfo.cb := Sizeof(StartupInfo); StartupInfo.dwFlags := STARTF_USESHOWWINDOW or STARTF_USESTDHANDLES; StartupInfo.hStdInput := ReadPipe1; StartupInfo.hStdOutput := WritePipe2; StartupInfo.wShowWindow := SW_HIDE; // CreateProcess [...] // read Win32Check( ReadFile( ReadPipe1, // source (@outputBuffer[1])^, // buffer-pointer PIPE_BUFFER_SIZE, // size bytesRead, // returns bytes actually read nil // overlapped on default )); // send Win32Check( WriteFile( WritePipe2, (@msg[1])^, // lpBuffer - workarround to avoid type cast NumberOfBytesToWrite, bytesWritten, // lpNumberOfBytesWritten nil // Overlapped )); 

Answer 2

Perhaps you can use named events for IPC signaling. They work fine in Win7, etc., When they are local (TEvent.Create ('local \ myserver'); When you need to do IPC between different sessions (for example, the client application and the background window), you need more rights and t .d. (By default, global \ cannot be used in win7 due to UAC?). http://docs.embarcadero.com/products/rad_studio/delphiAndcpp2009/HelpUpdate2/EN/html/devwin32/threadswaitingforatasktobecompleted_xml.html

For example: create an event for each connection (generated name for each connection).

Or take a look at another IPC implementation called pipe + events: https://micksmix.wordpress.com/2011/06/27/named-pipes-unit-for-delphi/

By the way: did you mention that you used profiling, but couldn’t say what takes the most time? What profiling did you use? Not a “profiler” like AQtime (http://smartbear.com/products/free-tools/aqtime-standard) or AsmProfiler (http://code.google.com/p/asmprofiler/)?

Answer 3

And a simple improvement could be: first send the number of bytes to send (so that the recipient knows how much data he can expect), and then send the data