How to transfer files larger than 2 147 483 646 bytes (~ 2 gigabytes) using Win32 TransmitFile ()?

Question

Quote from the MSDN Record for TransmitFile :

The maximum number of bytes that can be transferred using a single call to the TransmitFile function is 2,147,483,646, the maximum value for a 32-bit integer is minus 1. The maximum number of bytes to be sent in one call includes any transferred data before or after the file data for which specifies the lpTransmitBuffers parameter, plus the value specified in the nNumberOfBytesToWrite parameter for the length of the data files to be sent. If the application must transfer a file larger than 2,147,483,646 bytes, then each call transition can use several calls to the TransmitFile function not more than 2,147,483,646 bytes. Setting the nNumberOfBytesToWrite parameter to zero for a file larger than 2,147,483,646 bytes will also not work, because in this case the TransmitFile function will use the file size as a value for the number of bytes transferred.

Good. Sending a file of size 2*2,147,483,646 bytes (~ 4 GiB) using TransmitFile should then be split into at least two parts (for example, 2 GiB + 2 GiB in two TransmitFile calls). But how exactly could this be done, although it is preferable to also maintain a basic TCP connection between them?

When the file is really <= 2,147,483,646 bytes, you can simply write:

 HANDLE fh = CreateFile(filename, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL | FILE_FLAG_SEQUENTIAL_SCAN, NULL); TransmitFile(SOCK_STREAM_socket, fh, 0, 0, NULL, NULL, TF_DISCONNECT); 

to allow Windows to handle all lower-level elements (caching, splitting data into parts for efficient transfer, etc. However, unlike the comparable Linux sendfile () syscall, there is no obvious offset argument in the call (although the fifth argument, LPOVERLAPPED lpOverlapped , probably exactly what I'm looking for.) I suppose I could hack something, but I'm also looking at an elegant, effective Win32 solution from a person who really knows about it.

You can use the lpOverlapped parameter to specify the 64-bit offset in the file from which you can start transferring file data by setting the Offset and OffsetHigh member of the OVERLAPPED structure. If lpOverlapped is a NULL pointer, data transfer always starts with the current byte offset in the file.

So, due to the lack of a minimal example, easily accessible on the network, what challenges are needed to complete such a task?

Answer 1

I managed to find out, based on the comments.

So, if LPOVERLAPPED lpOverlapped is a null pointer, the call starts transferring at the current file file offset (as is the case with syscall Linux sendfile() and its off_t *offset parameter). This offset (pointer) can be manipulated using SetFilePointerEx , so you can write:

 #define TRANSMITFILE_MAX ((2<<30) - 1) LARGE_INTEGER total_bytes; memset(&total_bytes, 0, sizeof(total_bytes)); while (total_bytes < filesize) { DWORD bytes = MIN(filesize-total_bytes, TRANSMITFILE_MAX); if (!TransmitFile(SOCK_STREAM_socket, fh, bytes, 0, NULL, NULL, 0)) { /* error handling */ } total_bytes.HighPart += bytes; SetFilePointerEx(fh, total_bytes, NULL, FILE_BEGIN); } closesocket(SOCK_STREAM_socket); 

to complete the task. Not very elegant imo, but it works.