How to create another process and write output in python?

Question

I am just learning Python, but have 16 years experience with PERL and PHP.

I am trying to get ngrep output and write it to a log file using Python, as well as the tail of the log file. I saw several examples on the Internet, but some of them seem old and outdated, while others use shell = True, which is discouraged.

In perl, I just use something similar to the following

#!/usr/bin/perl
open(NGFH,"ngrep -iW byline $filter");
while ($line = <NGFH>) {
    open(LOG,">> /path/to/file.log")
    // highlighting, filtering, other sub routine calls
    print LOG $line
}

I have a tail, but ngrep does not. I would like to be able to run this endlessly and output the stream from ngrep to the log file after filtering. I could not get the output of ngrep for display in stdout, so as far as I understood. I expected to see the tail of the data file since the log file was updated and see the output from ngrep. For now, I just used bash to run the following.

echo "." >> /path/to/ngrep.log

Thanks!

Here is what I got so far ...

Update Now it works. I would not know how to improve it.

import subprocess
import select
import re

log = open('/path/to/ngrep.log','a+',0)
print log.name

n = subprocess.Popen(['ngrep', '-iW', 'byline'],\
    stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
p = select.poll()
p.register(n.stdout)

f = subprocess.Popen(['tail','-F','-n','0','/path/to/tailme.log'],\
    stdout=subprocess.PIPE,stderr=subprocess.PIPE)
p2 = select.poll()
p2.register(f.stdout)

def srtrepl(match):
    if match.group(0) == 'x.x.x.x':
        # do something
    if match.group(0) == 'x.x.y.y':
        # do something else

    return '\033[92m'+ match.group(0) + '\033[0m'

while True:
    if p.poll(1):
        line = n.stdout.readline()
        s = re.compile(r'(8.8.(4.4|8.8)|192.168.[0-9]{1,3}.[0-9]{1,3})' )
        print s.sub( srtrepl, line )
        log.write(n.stdout.readline())

    if p2.poll(1):
        print f.stdout.readline().rstrip('\n')

Answer 1

To emulate your perl code in Python:

#!/usr/bin/env python3
from subprocess import Popen, PIPE

with Popen("ngrep -iW byline".split() + [filter_], stdout=PIPE) as process, \
     open('/path/to/file.log', 'ab') as log_file:
    for line in process.stdout: # read b'\n'-separated lines
        # highlighting, filtering, other function calls
        log_file.write(line)

ngrep process filter_ , Python. . Python: subprocess.communicate() ( : , ngrep --line-buffered, grep, tail file.log, buffering=1 open(), ( ) log_file.flush() log_file.write(line)).

ngrep Python .

---

(ngrep, tail ), , , , async.io.