Fclose () / pclose () may block some file pointers

Question

The call to fclose() here after dup() with its file descriptor is blocked until the end of the child process (presumably because the thread has ended).

 FILE *f = popen("./output", "r"); int d = dup(fileno(f)); fclose(f); 

However, by manually executing pipe() , fork() , execvp() popen() , and then dup() using the channel read file descriptor, closing the original is not blocked.

 int p[2]; pipe(p); switch (fork()) { case 0: { char *argv[] = {"./output", NULL}; close(p[0]); dup2(p[1], 1); execvp(*argv, argv); } default: { close(p[1]); int d = dup(p[0]); close(p[0]); } } 

Why is this happening, and how can I close the FILE * returned from popen() and use the file descriptor in its place?

Update:

I know that the documentation says to use pclose() , however fclose() blocks. Also, I poked into glibc code, and pclose() just calls fclose() . The behavior is the same whether fclose() or pclose() .

Answer 1

Frustrated by the general answers so far (I can RTFM , tyvm), I have carefully examined this, step by step through and reading the glibc source.

In glibc, pclose() directly calls fclose() with no additional effect, so the 2 calls are the same. In fact, you could use pclose() and fclose() interchangeably. I'm sure this is just a coincidence in the evolutionary implementation, and using pclose() to close the FILE * returned from popen() still recommended.

The magic is in popen() . FILE * in glibc contains a jump table with pointers to the appropriate functions to handle calls such as fseek() , fread() and the relevance of fclose() . Calling popen() uses a different jump table than the one used by fopen() . The close element in this jump table points to the special function _IO_new_proc_close , which calls waitpid() on the pid stored in the area pointed to by FILE * .

Here is the corresponding call stack in my version of glibc, which I annotated with notes on what was going on:

 // linux waitpid system call interface #0 0x00f9a422 in __kernel_vsyscall () #1 0x00c38513 in __waitpid_nocancel () from /lib/tls/i686/cmov/libc.so.6 // removes fp from a chain of proc files // and waits for the process of the stored pid to terminate #2 0x00bff248 in _IO_new_proc_close (fp=0x804b008) at iopopen.c:357 // flushes the stream and calls close in its jump table #3 0x00c09ff3 in _IO_new_file_close_it (fp=0x804b008) at fileops.c:175 // destroys the FILEs buffers #4 0x00bfd548 in _IO_new_fclose (fp=0x804b008) at iofclose.c:62 // calls fclose #5 0x00c017fd in __new_pclose (fp=0x804b008) at pclose.c:43 // calls pclose #6 0x08048788 in main () at opener.c:34 

Thus, when using popen() returned FILE * should not be closed, even if you dup() its file descriptor, because it will be blocked until the child process completes. Of course, after that you will be left with a file descriptor in the channel, which will contain everything that the child process could write () before it was completed.

Not fread() with the file pointer returned with popen() , the base channel will not be affected, it is safe to use the file descriptor fileno() and will end by calling pclose() .

Answer 2

http://linux.die.net/man/3/popen

The pclose () function waits for the completion of the associated process and returns the exit status of the command returned by wait4 ().

Since pclose () wants to return the exit status, it needs to wait until the child finishes and generates it. Since fclose () calls pclose (), for fclose () the same applies to you.

If you use fork and exec and do the rest yourself, you do not call pclose () (directly or indirectly), so do not expect any time soon. Please note that if your program does not ignore SIGCHLD, your process will not end (instead it will be a zombie) until the child does. But at least your expenses will be made to exit first.

Answer 3

FILE* returned by popen() should be closed by pclose() , not fclose() . Then the documentation for pclose() states:

The pclose () function waits for the associated process to terminate and returns the command exit status as returned by wait4 ().

Thus, waiting is one of two things pclose() , other than closing the file descriptor. close() does only one thing: it closes the handle.

In answer to your second question, I think you can use the handle returned by fileno() . There you do not need dup() it. After you are done with it, pclose() original.