Can you not give up?

Question

Can you not give up?

In a program written in C ++ and compiled using MinGW-w64 under windows, I look at several files at the same time in separate streams. Since file names may have non-ASCII characters, I cannot use the standard C ++ library std::ifstream, since it does not support file names wchar. Therefore, I need to use the C library _wfopenfrom the Win32 API.

However, I get a very strange error that I reproduced in MCVE. After reading n bytes with fread (), the result of _ftelli64sometimes does not increase by n, but by several bytes less or more.

With reading a single thread, the problem disappeared, but with the help std::ifstream.

It acts as if fread has a race condition that would not be reentrant then.

In the following example, I replaced _wfopenwith fopen, because the error still exists.

#include <iostream>
#include <vector>
#include <string>
#include <sstream>
#include <fstream>
#include <thread>

constexpr const int numThreads = 8;
constexpr const int blockSize = 65536+8;
constexpr const int fileBlockCount = 48; //3MB files

void readFile(const std::string & path)
{
    std::cout << "Reading file " << path << "\n";
    std::vector<char> buffer(blockSize);

    FILE * f = fopen(path.c_str(), "rb");
    for(int i=0;i<fileBlockCount;++i)
    {

        int64_t pos_before = _ftelli64(f);
        int64_t n = fread(buffer.data(), 1, buffer.size(),f);
        int64_t pos_after = _ftelli64(f);
        int64_t posMismatch = (int64_t)pos_after-(pos_before+n);
        if(ferror(f))
        {
            std::cout << "fread error\n";
        }
        if(posMismatch!=0)
        {
            std::cout << "Error " << path
                  << " / ftell before " << pos_before
                  << " / fread returned " << n
                  << " / ftell after " << pos_after
                  << " / mismatch " << posMismatch << "\n";
        }
    }
    fclose(f);
}

int main()
{
    //Generate file names
    std::vector<std::string> fileNames(numThreads);
    for(int i=0;i<numThreads;++i)
    {
        std::ostringstream oss;
        oss << i << ".dat";
        fileNames[i] = oss.str();
    }


    //Create dummy data files
    for(int i=0;i<numThreads;++i)
    {
        std::ofstream f(fileNames[i], std::ios_base::binary);
        for(int j=0;j<blockSize*fileBlockCount;++j)
        {
            f.put((char)(j&255));
        }
    }


    //Read data files in separate threads
    std::vector<std::thread> threads;
    for(int i=0;i<numThreads;++i)
    {
        threads.emplace_back(readFile, fileNames[i]);
    }

    //This waits for the threads to finish
    for(int i=0;i<numThreads;++i)
    {
        threads[i].join();
    }
    threads.clear();

    std::cout << "Done";
}

The output randomly looks like this:

Error 3.dat / ftell before 65544 / fread returned 65544 / ftell after 131089 / mismatch 1
Error 7.dat / ftell before 0 / fread returned 65544 / ftell after 65543 / mismatch -1
Error 7.dat / ftell before 65543 / fread returned 65544 / ftell after 131088 / mismatch 1
Error 3.dat / ftell before 2162953 / fread returned 65544 / ftell after 2228498 / mismatch 1
Error 7.dat / ftell before 2162952 / fread returned 65544 / ftell after 2228497 / mismatch 1
Error 3.dat / ftell before 3080570 / fread returned 65544 / ftell after 3146112 / mismatch -2
Error 7.dat / ftell before 3080569 / fread returned 65544 / ftell after 3146112 / mismatch -1
Error 2.dat / ftell before 65544 / fread returned 65544 / ftell after 131089 / mismatch 1
Error 6.dat / ftell before 0 / fread returned 65544 / ftell after 65543 / mismatch -1
Error 6.dat / ftell before 65543 / fread returned 65544 / ftell after 131088 / mismatch 1
Error 2.dat / ftell before 2162953 / fread returned 65544 / ftell after 2228498 / mismatch 1
Error 6.dat / ftell before 2162952 / fread returned 65544 / ftell after 2228497 / mismatch 1
Error 2.dat / ftell before 3080570 / fread returned 65544 / ftell after 3146112 / mismatch -2
Error 6.dat / ftell before 3080569 / fread returned 65544 / ftell after 3146112 / mismatch -1