Using text file as stdin in python under windows 7

Question

I am a win7 user.

I accidentally read about redirects (e.g. command1 < infile > outfile ) on * nix systems, and then discovered that something similar could be done on Windows (link) . And python can also do something similar using pipes (?) Or stdin / stdout (?).

I do not understand how this happens on Windows, so I have a question.

I am using some kind of proprietary windows program (.exe). This program can add data to a file. For simplicity, suppose this is the equivalent of something like

 while True: f = open('textfile.txt','a') f.write(repr(ctime()) + '\n') f.close() sleep(100) 

Question: Can I use this file (textfile.txt) as stdin? I mean that the script (while it is running) should always (more than once) process all new data, i.e.

In the "endless loop":

• The program (.exe) is writing something.

• Python script captures data and processes.

Could you write how to do this in python, or perhaps in win cmd / .bat or something else.

This is insanely cool. I want to learn this !: D

Answer 1

If I read your question correctly, you want to pass the output from one command to another.

This is usually done as follows:

 cmd1 | cmd2 

However, you say that your program writes only files. I would double check the documentation to make sure that this is not a way to force the command to write to stdout instead of a file.

If this is not possible, you can create a so-called named pipe . It is displayed as a file in your file system, but in fact it is just a data buffer that can be written and read (data is a stream and can be read only once). This means that your program, after reading it, will not be completed until the program writing to the handset stops writing and closes the "file". I have no experience with named pipes in windows, so you need to ask a new question. One bottom side of the pipes is that they have a limited buffer size. Therefore, if there is no data on reading the program from the channel, then after the buffer is full, the write program cannot continue and just wait indefinitely until the program starts reading from the channel.

An alternative is that on Unix there is a program called tail , which can be configured to continuously monitor the file for changes and output any data as it is added to the file (with a slight delay.

 tail --follow=textfile.txt --retry | mycmd # wait for data to be appended to the file and output new data to mycmd cmd1 >> textfile.txt # append output to file 

One note about this is that tail will not stop just because the first command stopped writing to the file. tail will continue to listen to changes to this file forever, either until mycmd stops listening to tail , or until tail is killed (or "sigint-ed").

This question has various answers on how to get the tail version on a Windows machine.

Answer 2

 import sys sys.stdin = open('textfile.txt', 'r') for line in sys.stdin: process(line) 

Answer 3

If the program writes to textfile.txt , you cannot change this to redirect to the stdin your Python script unless you recompile the program for this.

If you needed to edit the program, you need to write to stdout , not a file in the file system. That way, you can use redirection operators to pass it to your Python script (in your case, the | operator).

Assuming you cannot do this, you can write a program that polls the changes in a text file and consumes only the new recorded data, keeping track of how much it read the last time it was updated.

Answer 4

When you use < to direct the output of a file to a python script, this script receives this data on it stdin stream .

Just read from sys.stdin to get this data:

 import sys for line in sys.stdin: # do something with line