The difference between fflush and fsync

Question

I thought fsync() does fflush() internally, so using fsync() in a stream is fine. But I get an unexpected result when executed under network I / O.

My code snippet:

 FILE* fp = fopen(file, "wb"); /* multiple fputs() calls like: */ fputs(buf, fp); ... ... fputs(buf.c_str(), fp); /* get fd of the FILE pointer */ fd = fileno(fp); #ifndef WIN32 ret = fsync(fd); #else ret = _commit(fd); fclose(fp); 

But it looks like _commit() not _commit() data (I tried on Windows and the data was written to the exported Linux file system).

When I changed the code to be:

 FILE* fp = fopen(file, "wb"); /* multiple fputs() calls like: */ fputs(buf, fp); ... ... fputs(buf.c_str(), fp); /* fflush the data */ fflush(fp); fclose(fp); 

this flushes the data.

I am wondering if _commit() does the same thing as fflush() . Any inputs?

Answer 1

fflush() runs on FILE* , it just clears the internal buffers in FILE* your application from the OS.

fsync runs at a lower level, it tells the OS to flush its buffers to physical media.

The OS strongly caches the data you write to the file. If the OS enforces each entry to get to disk, everything will be very slow. fsync (by the way) lets you control when data should go to disk.

In addition, fsync / commit works with a file descriptor. He does not know about FILE* and cannot flush his buffers. FILE* lives in your application, file descriptors live in the kernel of the OS, as a rule.

Answer 2

The standard C function fflush() and the POSIX fsync() system call are conceptually somewhat similar. fflush() works with C file streams ( FILE objects) and is therefore portable. fsync() works with POSIX file descriptors. Both cause the sending of buffered data to the destination.

On a POSIX system, each C file stream has a associated file descriptor , and all operations on the C file stream will be implemented by delegating, if necessary, POSIX system calls that operate on the file descriptor.

You might think that calling fflush on a POSIX system will write any data to the file stream buffer, then calling fsync() on the file descriptor for that file stream. Thus, on a POSIX system, it would not be necessary to follow the fflush call using the fsync(fileno(fp)) call. But is this so: is there a fsync call from fflush ?

No, calling fflush on a POSIX system does not mean that fsync will be called.

C standard for fflush says (highlighted)

causes any unwritten data for stream [] to be transferred to the host environment for writing to a file

The statement that data should be written, not written, implies that further buffering by the host environment is allowed. This buffering by the host environment may include internal buffering for the POSIX environment that fsync flushes. So, a careful reading of the C standard suggests that the standard does not require the implementation of POSIX fsync .

The standard description of fflush POSIX does not declare, as an extension of the semantics of C , that fsync is invoked.

Answer 3

I could say for simplicity:

use fsync() with non-stream files (integer file descriptors)

use fflush() with file streams.

Also here is help from a person:

 int fflush(FILE *stream); // flush a stream, FILE* type int fsync(int fd); // synchronize a file in-core state with storage device // int type 

Answer 4

To force the latest changes to be used on the drive, use the sync () or fsync () functions.

fsync () synchronizes all file data and metadata with read-only memory. It must be called immediately before closing the corresponding file.

sync () will transfer all modified files to disk.

Answer 5

I think the below document from python ( https://docs.python.org/2/library/os.html ) clarifies it very well.

os.fsync (fd) Force writing a file from filedescriptor fd to disk. On Unix, this calls the built-in fsync () function; on Windows, MS _commit ().

If you start with the Python file object f, first execute f.flush () and then execute os.fsync (f.fileno ()) to make sure that all the internal buffers associated with f are written to disk.

Availability: Unix and Windows since 2.2.3.

Answer 6

fflush() and fsync() can be used to check and write data to the media (but this is not always possible):

1. first use fflush(fp) in the output stream ( fp is FILE * obtained from fopen or one of the standard stdout or stderr streams) to write the contents of the buffer associated with the stream to the OS.
2. then use fsync(fileno(fp)) to tell the OS to write its own buffers to the media.

However, note that fileno() and fsync() are POSIX functions that may not be available on all systems, especially legacy Microsoft systems, where alternatives may be called _fileno() , _fsync() or _commit() ...