Failed to read the last few kilobytes of logical drives in Windows 7 64-bit

Question

I have problems with logical drives. For clarity, my definition of “Physical Disk” (PD) is a raw disk, regardless of partitioning. A “logical drive” (LD) refers to, for example, an E :, drive F: etc.

Using examples from RRUZ (a member of my hero SO) and implementing the WMI class , I created the Freepascal program for reading disks. I access PD using \. \ PhyscialDiskX, and this works great with the examples created by RRUZ ( here ). I can read all bytes without problems for PD.

I use the same descriptor technique for logical volumes that are \? \ E: or \? \ F: etc. Then I use IOCTL_DISK_GET_LENGTH_INFO to get the length of PD or LV, and then read the range of bytes until ReadBytes = Total Length. I read on the MSDN website that it will automatically get the size of any device descriptor to which it is transferred - both PD and LD. Indeed, I checked the szie values ​​returned by my againzt WinHex program, FTK Imager, HxD, and several other low-level disk tools. With the exception of 1-byte variances caused by zero or 1 start positions, they coincide.

However, for some reason, my program cannot get the final 32Kb on the 64-bit version of Windows 7 Pro, even though it runs as an administrator. It reads the entire disk, and then in the last buffer read (which runs as 64 KB buffers) BytesRead returns -1. Using a debugger, I developed the following values:

493,846,527 exact LV size of Drive F: 493,813,760 total bytes read at time of failure 32,767 bytes missing 

Result of the following

 BytesRead := FileRead(hDiskHandle, Buffer, (DiskSize - TotalBytesRead)); 

equals -1 on the last buffer read. This is the line that checks the end of the disk, saying: "if the number remaining until reading is less than the size of the buffer, try just reading what is left." Thus, the byte value that you want to save in FileRead at the end is 32,767 (since DiskSize - TotalBytesRead at this point is 32,767, which means how many bytes are left to read on the disk). The assigned buffer size is 64 KB. I understand that you can put less in the buffer than it can hold but no more (FileRead says: “The buffer must be at least the number of bytes in length. Does this check”? Is this correct? If it is not, is it may be (and probably a problem).

I do not know if this is due to IOCTL_DISK_GET_LENGTH_INFO, buffer storage or something else? Hope someone can help? I also posted along with some screenshots on the Lazarus Freepascal forum. Here are my relevant code sections:

A pen:

 // Create handle to source disk. Abort if fails hSelectedDisk := CreateFileW(PWideChar(SourceDevice), FILE_READ_DATA, FILE_SHARE_READ, nil, OPEN_EXISTING, FILE_FLAG_SEQUENTIAL_SCAN, 0); if hSelectedDisk = INVALID_HANDLE_VALUE then begin RaiseLastOSError; end 

Calculate the dimensional bytes of the ions of this device:

 ExactDiskSize := GetDiskLengthInBytes(hSelectedDisk); 

Now read the device and save the input as a flat file

 ImageResult := WindowsImageDisk(hSelectedDisk, ExactDiskSize, HashChoice, hImageName); 

Functions for the above:

 function GetDiskLengthInBytes(hSelectedDisk : THandle) : Int64; const // These are defined at the MSDN.Microsoft.com website for DeviceIOControl // and https://forum.tuts4you.com/topic/22361-deviceiocontrol-ioctl-codes/ { IOCTL_DISK_GET_DRIVE_GEOMETRY = $0070000 IOCTL_DISK_GET_PARTITION_INFO = $0074004 IOCTL_DISK_SET_PARTITION_INFO = $007C008 IOCTL_DISK_GET_DRIVE_LAYOUT = $007400C IOCTL_DISK_SET_DRIVE_LAYOUT = $007C010 IOCTL_DISK_VERIFY = $0070014 IOCTL_DISK_FORMAT_TRACKS = $007C018 IOCTL_DISK_REASSIGN_BLOCKS = $007C01C IOCTL_DISK_PERFORMANCE = $0070020 IOCTL_DISK_IS_WRITABLE = $0070024 IOCTL_DISK_LOGGING = $0070028 IOCTL_DISK_FORMAT_TRACKS_EX = $007C02C IOCTL_DISK_HISTOGRAM_STRUCTURE = $0070030 IOCTL_DISK_HISTOGRAM_DATA = $0070034 IOCTL_DISK_HISTOGRAM_RESET = $0070038 IOCTL_DISK_REQUEST_STRUCTURE = $007003C IOCTL_DISK_REQUEST_DATA = $0070040 IOCTL_DISK_CONTROLLER_NUMBER = $0070044 IOCTL_DISK_GET_PARTITION_INFO_EX = $0070048 IOCTL_DISK_SET_PARTITION_INFO_EX = $007C04C IOCTL_DISK_GET_DRIVE_LAYOUT_EX = $0070050 IOCTL_DISK_SET_DRIVE_LAYOUT_EX = $007C054 IOCTL_DISK_CREATE_DISK = $007C058 IOCTL_DISK_GET_LENGTH_INFO = $007405C // Our constant... SMART_GET_VERSION = $0074080 SMART_SEND_DRIVE_COMMAND = $007C084 SMART_RCV_DRIVE_DATA = $007C088 IOCTL_DISK_GET_DRIVE_GEOMETRY_EX = $00700A0 IOCTL_DISK_UPDATE_DRIVE_SIZE = $007C0C8 IOCTL_DISK_GROW_PARTITION = $007C0D0 IOCTL_DISK_GET_CACHE_INFORMATION = $00740D4 IOCTL_DISK_SET_CACHE_INFORMATION = $007C0D8 IOCTL_DISK_GET_WRITE_CACHE_STATE = $00740DC IOCTL_DISK_DELETE_DRIVE_LAYOUT = $007C100 IOCTL_DISK_UPDATE_PROPERTIES = $0070140 IOCTL_DISK_FORMAT_DRIVE = $007C3CC IOCTL_DISK_SENSE_DEVICE = $00703E0 IOCTL_DISK_INTERNAL_SET_VERIFY = $0070403 IOCTL_DISK_INTERNAL_CLEAR_VERIFY = $0070407 IOCTL_DISK_INTERNAL_SET_NOTIFY = $0070408 IOCTL_DISK_CHECK_VERIFY = $0074800 IOCTL_DISK_MEDIA_REMOVAL = $0074804 IOCTL_DISK_EJECT_MEDIA = $0074808 IOCTL_DISK_LOAD_MEDIA = $007480C IOCTL_DISK_RESERVE = $0074810 IOCTL_DISK_RELEASE = $0074814 IOCTL_DISK_FIND_NEW_DEVICES = $0074818 IOCTL_DISK_GET_MEDIA_TYPES = $0070C00 } IOCTL_DISK_GET_LENGTH_INFO = $0007405C; type TDiskLength = packed record Length : Int64; end; var BytesReturned: DWORD; DLength: TDiskLength; ByteSize: int64; begin BytesReturned := 0; // Get the length, in bytes, of the physical disk if not DeviceIOControl(hSelectedDisk, IOCTL_DISK_GET_LENGTH_INFO, nil, 0, @DLength, SizeOf(TDiskLength), BytesReturned, nil) then raise Exception.Create('Unable to determine byte capacity of disk.'); ByteSize := DLength.Length; ShowMessage(IntToStr(ByteSize)); result := ByteSize; end; 

Disk read function

 function WindowsImageDisk(hDiskHandle : THandle; DiskSize : Int64; HashChoice : Integer; hImageName : THandle) : Int64; var Buffer : array [0..65535] of Byte; // 1048576 (1Mb) or 262144 (240Kb) or 131072 (120Kb buffer) or 65536 (64Kb buffer) BytesRead : integer; NewPos, SectorCount, TotalBytesRead, BytesWritten, TotalBytesWritten : Int64; ... // Now to seek to start of device FileSeek(hDiskHandle, 0, 0); repeat // Read device in buffered segments. Hash the disk and image portions as we go if (DiskSize - TotalBytesRead) < SizeOf(Buffer) then begin // Read 65535 or less bytes BytesRead := FileRead(hDiskHandle, Buffer, (DiskSize - TotalBytesRead)); BytesWritten := FileWrite(hImageName, Buffer, BytesRead); end else begin // Read 65536 (64kb) at a time BytesRead := FileRead(hDiskHandle, Buffer, SizeOf(Buffer)); BytesWritten := FileWrite(hImageName, Buffer, BytesRead); end; if BytesRead = -1 then begin ShowMessage('There was a read error encountered. Aborting'); // ERROR IS THROWN AT THIS POINT ONLY WITH LD - not PD's exit; end else begin inc(TotalBytesRead, BytesRead); inc(TotalBytesWritten, BytesWritten); NewPos := NewPos + BytesRead; ... until (TotalBytesRead = DiskSize); 

Answer 1

This is probably a border validation error. Quote from MSDN ( CreateFile , pay attention to opening physical disks and volumes that you call logical disks):

To read or write to the last few sectors of a volume, you must call DeviceIoControl and specify FSCTL_ALLOW_EXTENDED_DASD_IO

Answer 2

I suspect that the problem is using 64-bit integers and arithmetic to calculate the value passed as a 32-bit integer:

 FileRead(hDiskHandle, Buffer, (DiskSize - TotalBytesRead)); 

I can’t explain with certainty why this can only affect LD, not PD, except it is assumed that there might be some difference in the declared DiskSize , which somehow avoids Int64 arithmetic / 32-bit problem in this case.

eg. If the 32-bit truncated result of Int64 arithmetic is NEGATIVE (you only need to set the high bit, i.e. 1 is not 0), then FileRead () will return -1 , since a negative value for "bytes to read" is invalid.

But if the high bit is NOT set as a result, which leads to a positive value, then even if this value is significantly greater than 64 KB, this will not cause an error, since this call is called only when you have already determined that less than 64 is required for reading Kbytes of bytes. Int64 32-bit truncated arithmetic may cause the request to read 2 BILLION bytes, but FileRead () will only read the actual 32K bytes, which remain anyway.

However, this fact indicates a solution (provided that this diagnosis is correct).

As already noted, FileRead () (which is just the ReadFile () shell on Windows) will read either the number of bytes or the number of remaining bytes to be read, whichever is lower .

So, if you specify 64 KB, but only 32 KB remains, then only 32 KB will be read.

You can replace all this code:

 if (DiskSize - TotalBytesRead) < SizeOf(Buffer) then begin // Read 65535 or less bytes BytesRead := FileRead(hDiskHandle, Buffer, (DiskSize - TotalBytesRead)); BytesWritten := FileWrite(hImageName, Buffer, BytesRead); end else begin // Read 65536 (64kb) at a time BytesRead := FileRead(hDiskHandle, Buffer, SizeOf(Buffer)); BytesWritten := FileWrite(hImageName, Buffer, BytesRead); end; 

Just:

 BytesRead := FileRead(hDiskHandle, Buffer, SizeOf(Buffer)); BytesWritten := FileWrite(hImageName, Buffer, BytesRead); 

This eliminates 64-bit arithmetic and any chance of errors that result from 64-bit results being transmitted in 32-bit values. If the end segment contains only 32 KB, then only 32 KB will be read.

You can also simplify the end of the loop (removing 64-bit arithmetic, unless you need the accumulated values ​​for other purposes). Instead of accumulating the total number of bytes read, you can end the loop only when your FileRead () reads less than the specified number of bytes. i.e. BytesRead <64KB

Even if your drive is an exact multiple of 64KB blocks, your penultimate FileRead () will return a full 64K buffer, and the next next FileRead () will read bytes 0 , which is <; 64K, completing the loop. :)

Answer 3

32,767 is an odd number, not a multiple of the size of the sector. This means that the sector of the final part is simply not readable.

Answer 4

Ok, I did it.

Credit to the user 2024154, as this was the first important thing. Therefore, based on this, I gave an answer.

However, it is unclear how to correctly assign its value to const. After many hours of Googling, I came across this one . This was the only Delphi example I could find where it actually shows FSCTL_ALLOW_EXTENDED_DASD_IO, although they all had to look through most to pull the values ​​together.

In the interest of someone else, the values ​​that I need, which now work:

 const FILE_DEVICE_FILE_SYSTEM = $00000009; FILE_ANY_ACCESS = 0; METHOD_NEITHER = 3; FSCTL_ALLOW_EXTENDED_DASD_IO = ((FILE_DEVICE_FILE_SYSTEM shl 16) or (FILE_ANY_ACCESS shl 14) or (32 shl 2) or METHOD_NEITHER); 

Then I used FSCTL_ALLOW_EXTENDED_DASD_IO after the first descriptor creation, and then:

  if not DeviceIOControl(hSelectedDisk, FSCTL_ALLOW_EXTENDED_DASD_IO, nil, 0, nil, 0, BytesReturned, nil) then raise Exception.Create('Unable to initiate FSCTL_ALLOW_EXTENDED_DASD_IO disk access.'); 

This works with freepascal and with some minor tweaks should work with Delphi.

Thank you all for your continued help, especially user2024154 and, as always, David for his continued help.

UPDATE: Except when physical disk access does not work at all! But I will do some work.