Does the pipeline in SET / P fail due to an uninitialized data pointer?

Question

Suppose we have a sample.txt text file:

 one two ... 

Now we want to delete the first line:

 two ... 

A quick way to do this is to use input redirection, set /P and findstr ¹ (I know there are other ways to use more or for /F , but let's forget about them for now):

 @echo off < "sample.txt" ( set /P ="" findstr "^" ) 

The output will be as expected.

However, why the output is empty when I replace the input redirection < with type and channel | :

 @echo off type "sample.txt" | ( set /P ="" findstr "^" ) 

When I replace set /P ="" with pause > nul , the output is what I expect - the output file is output, but with the first character of the first line missing (since it is consumed by pause ). But why does set /P seem to consume everything, not just the first line, as it does with < redirects? This is mistake?

It seems to me that set /P fails to adequately initialize the read pointer for data transmitted over the channels.

I watched this strange behavior in Windows 7 and Windows 10.

---

This becomes even stranger: when you call a script that contains this channel several times, for example, with a loop like for /L %I in (1,1,1000) do @pipe.bat , and the input file contains about fifteen lines or more, sometimes (several times out of a thousand) a fragment of the input file is returned; this fragment is exactly the same every time; It seems that at the beginning of 80 bytes is missing.

---

<sub> 1) findstr freezes if the last line does not end with a line break, so suppose it is. Sub>

Answer 1

When retrieving data, set /p tries to fill a buffer with 1023 characters (if available) with data from stdin. Upon completion of this read operation, the first end of the line is searched, and as soon as it is found (or the end of the buffer is reached), the SetFilePointer API SetFilePointer called to move the input stream pointer after the end, read the line. Thus, the next read operation will begin to output data after the read line.

This works flawlessly when the disk file is associated with the input stream, but as Microsoft states in the SetFilePointer documentation

The hFile parameter must refer to a file stored on the search appliance; for example, disk capacity. Calling the SetFilePointer function by using a call to a non-searching device, such as a pipe or communication device is not supported, although the SetFilePointer function may not return an error. The behavior of the SetFilePointer function in this case is undefined.

It happens that in the absence of any error, the call to reorder the read pointer is not made, when stdin is connected to the channel, the pointer does not move back and 1023 bytes (or the number of bytes read) read.

edited in response to an Aacini request

The set command is processed by the eSet function, which calls SetWork to determine what type of set command will be executed.

Since this is set /p , the SetPromptUser function is SetPromptUser , and from this function, the ReadBufFromInput function ReadBufFromInput called

 add esp, 0Ch lea eax, [ebp+var_80C] push eax ; int push 3FFh ; int lea eax, [ebp+Value] push eax ; int xor esi, esi push 0FFFFFFF6h ; nStdHandle mov word ptr [ebp+Value], si call edi ; GetStdHandle(x) ; GetStdHandle(x) push eax ; hFile call _ReadBufFromInput@16 ; ReadBufFromInput(x,x,x,x) 

it requests 3FFh characters (1023) from the standard input descriptor ( 0FFFFFFF6h = -10 = STD_INPUT_HANDLE )

ReadBufFromInput uses the GetFileType API to determine whether to read it from the console or from a file.

 ; Attributes: bp-based frame ; int __stdcall ReadBufFromInput(HANDLE hFile, int, int, int) _ReadBufFromInput@16 proc near hFile= dword ptr 8 ; FUNCTION CHUNK AT .text:4AD10D3D SIZE 00000006 BYTES mov edi, edi push ebp mov ebp, esp push [ebp+hFile] ; hFile call ds: __imp__GetFileType@4 ; GetFileType(x) and eax, 0FFFF7FFFh cmp eax, 2 jz loc_4AD10D3D 

and, since in this case it is a pipe ( GetFileType returns 3 ), the code goes to the ReadBufFromFile function

 ; Attributes: bp-based frame ; int __stdcall ReadBufFromFile(HANDLE hFile, LPWSTR lpWideCharStr, DWORD cchWideChar, LPDWORD lpNumberOfBytesRead) _ReadBufFromFile@16 proc near var_C= dword ptr -0Ch cchMultiByte= dword ptr -8 NumberOfBytesRead= dword ptr -4 hFile= dword ptr 8 lpWideCharStr= dword ptr 0Ch cchWideChar= dword ptr 10h lpNumberOfBytesRead= dword ptr 14h 

This function will call the ReadFile API ReadFile to retrieve the specified number of characters.

 push ebx ; lpOverlapped push [ebp+lpNumberOfBytesRead] ; lpNumberOfBytesRead mov [ebp+var_C], eax push [ebp+cchWideChar] ; nNumberOfBytesToRead push edi ; lpBuffer push [ebp+hFile] ; hFile call ds: __imp__ReadFile@20 ; ReadFile(x,x,x,x,x) 

The returned buffer is repeated in search of the end of the line, and as soon as it is found, the pointer in the input stream will be moved after the found binding

 .text:4AD06A15 loc_4AD06A15: .text:4AD06A15 cmp [ebp+NumberOfBytesRead], 3 .text:4AD06A19 jl short loc_4AD06A2D .text:4AD06A1B mov al, [esi] .text:4AD06A1D cmp al, 0Ah .text:4AD06A1F jz loc_4AD06BCF .text:4AD06A25 .text:4AD06A25 loc_4AD06A25: .text:4AD06A25 cmp al, 0Dh .text:4AD06A27 jz loc_4AD06D14 .text:4AD06A2D .text:4AD06A2D loc_4AD06A2D: .text:4AD06A2D movzx eax, byte ptr [esi] .text:4AD06A30 cmp byte ptr _DbcsLeadCharTable[eax], bl .text:4AD06A36 jnz loc_4AD12018 .text:4AD06A3C dec [ebp+NumberOfBytesRead] .text:4AD06A3F inc esi .text:4AD06A40 .text:4AD06A40 loc_4AD06A40: .text:4AD06A40 cmp [ebp+NumberOfBytesRead], ebx .text:4AD06A43 jg short loc_4AD06A15 .text:4AD06BCF loc_4AD06BCF: .text:4AD06BCF cmp byte ptr [esi+1], 0Dh .text:4AD06BD3 jnz loc_4AD06A25 .text:4AD06BD9 jmp loc_4AD06D1E .text:4AD06D14 loc_4AD06D14: .text:4AD06D14 cmp byte ptr [esi+1], 0Ah .text:4AD06D18 jnz loc_4AD06A2D .text:4AD06D1E .text:4AD06D1E loc_4AD06D1E: .text:4AD06D1E mov eax, [ebp+var_C] .text:4AD06D21 mov [esi+2], bl .text:4AD06D24 sub esi, edi .text:4AD06D26 inc esi .text:4AD06D27 inc esi .text:4AD06D28 push ebx ; dwMoveMethod .text:4AD06D29 push ebx ; lpDistanceToMoveHigh .text:4AD06D2A mov [ebp+cchMultiByte], esi .text:4AD06D2D add esi, eax .text:4AD06D2F push esi ; lDistanceToMove .text:4AD06D30 push [ebp+hFile] ; hFile .text:4AD06D33 call ds: __imp__SetFilePointer@16 ; SetFilePointer(x,x,x,x) 

Answer 2

A brief summary of the long discussion on dopid (already mentioned Aacini set / p problems with channels ).

Reading with set /p from a redirect always reads to the end of the line and removes the characters \r\n .

Reading with set /p from the channel, reads up to 1023 bytes from the buffer. It does not stop at any \r or \n characters, but it leaves all contents after \n .
After closing the pipe on the left side, set /p on the right side will read empty lines.