How to log everything that happens in a Python interactive shell session?

Question

I would like to have real-time access to the input and error of the interpreter, as well as to standard outputs. Preferably, this information will be written to a file so that I can query the file for changes after the entire interpreter command has been entered. For example, given an interpreter session:

>>> 5 * 7 35 >>> print("Hello, world!") Hello, world! >>> "Hello, world!" 'Hello, world!' 

I want to see the following in a log file:

 > 5 * 7 35 > print("Hello, world!") Hello, world! > "Hello, world!" 'Hello, world!' 

Formatting is not important; The important thing is that I can search for a file for keywords to trigger interactive events during a session.

What I have learned so far is trying to do this:

The Python code module allows me to create an InteractiveConsole object, the raw_input method, which I can override to enter a file, for example:

 import code class LoggedConsole(code.InteractiveConsole): def __init__(self, locals): super(LoggedConsole, self).__init__(locals) self.file = open('consolelog.dat', 'a') def __del__(self): self.file.close() def raw_input(self, prompt=""): data = input(prompt) self.file.write(data+'\n') return data 

In addition, InteractiveConsole uses the built-in write method to register errors, which I can override to:

 def write(self, data): sys.stderr.write(data) self.file.write(data+'\n') 

I also found out that the following snippet will write all stdout:

 class Tee(object): def __init__(self): self.file = open('consolelog.dat', 'a') self.stdout = sys.stdout def __del__(self): sys.stdout = self.stdout self.file.close() def write(self, data): self.file.write(data) self.stdout.write(data) sys.stdout = Tee() 

My (broken) attempt to combine all of this was to then create a LoggedConsole object and pass it Tee to locals.

 console = LoggedConsole(locals={sys.stdout:LoggedExec()}) console.interact() 

(I have not met local residents before, so maybe I am doing it wrong here, but I am not getting an error message.)

In any case, this will open a new interactive console and will log (after closing) all input and errors, but not output. For a moment I hit my head about it, and it seems to me that I'm close, but maybe not even.

Also, is there a way for all this to happen during the session? Currently, all logging occurs after the session is closed.

Thank you for your time, sorry for the wall of text.

edit: I would like to be able to accomplish this in a standard Python interpreter for portability purposes.

edit2: Jaime snippet works great for registering everything I need. In any case, what can I get him to do this in real time, rather than wait for the session to close?

Edit3: Figured it out :). Final working fragment:

 import code import sys class Tee(object): def __init__(self, log_fname, mode='a'): self.log = open(log_fname, mode) def __del__(self): # Restore sin, so, se sys.stdout = sys.__stdout__ sys.stdir = sys.__stdin__ sys.stderr = sys.__stderr__ self.log.close() def write(self, data): self.log.write(data) self.log.flush() sys.__stdout__.write(data) sys.__stdout__.flush() def readline(self): s = sys.__stdin__.readline() sys.__stdin__.flush() self.log.write(s) self.log.flush() return s def flush(foo): return sys.stdout = sys.stderr = sys.stdin = Tee('consolelog.dat', 'w') console = code.InteractiveConsole() console.interact() 

Answer 1

I tested this only in python2.7. I do not have 3 convenient.

 import code import sys class Tee(object): def __init__(self, log_fname, mode='a'): self.log = open(log_fname, mode) def __del__(self): # Restore sin, so, se sys.stdout = sys.__stdout__ sys.stdir = sys.__stdin__ sys.stderr = sys.__stderr__ self.log.close() def write(self, data): self.log.write(data) sys.__stdout__.write(data) def readline(self): s = sys.__stdin__.readline() self.log.write(s) return s # Tie the ins and outs to Tee. sys.stdout = sys.stderr = sys.stdin = Tee('consolelog.dat', 'w') console = code.InteractiveConsole() console.interact() 

Answer 2

take a look at IPython (did not use it yourself). Here's a section in the docs that might be of particular interest: http://ipython.org/ipython-doc/dev/interactive/reference.html#session-logging-and-restoring

Answer 3

See this Doug Hellmann Virtualenv article on how to register an iPython session:

If you are comfortable working with an interactive invitation this way, but want to record what you do for reference after closing the session, you can use the IPythons logging function to write the session to a file. To activate the log, use the %logstart control %logstart , as shown in Listing 5. The output file is the Python source file, so it is easy to clean it up and turn it into a “real” module when you finish experimenting.

 In [6]: %logstart Activating auto-logging. Current session state plus future input saved. Filename : ipython_log.py Mode : rotate Output logging : False Raw input log : False Timestamping : False State : active In [7]: a = 5 In [8]: b = 6 In [9]: c = a * b In [10]: c Out[10]: 30 In [11]: d = [ a, b, c] In [12]: d Out[12]: [5, 6, 30] In [13]: %logstop 

Answer 4

You can simply use the unix script command, try:

 script -a filename.txt python >> print("hi") hi >> exit() exit 

filename.txt will write down everything you did in this session, it will look something like this:

 Script started on Sat Dec 14 11:18:41 2013 python >> print('hi') hi >> exit() exit Script done on Sat Dec 14 11:18:59 2013 

Answer 5

You can try using my logging tool. This is not perfect yet, but he solved my problem, which seems similar to yours.

https://github.com/hholst80/loginteractive

It works using LD_PRELOAD to connect stdin.txt (or $ STDIN) to stdout. It works for python and octave, although I have not tested it yet.