How to read standard output of a Python script from within it?

Question

Here is my problem. I have an application which prints some traces to the standard output using logging module. Now, I want to be able to read those traces at the same time in order to wait for specific trace I need.

This is for the testing purpose. So for example, if wanted trace does not occur in about 2 seconds, test fails.

I know I can read output of another scripts by using something like this:

import subprocess
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
while True:
    line = p.stdout.readline()
    print line
    if line == '' and p.poll() != None:
        break

But, how can I do something similar from the script itself? Thanks in advance.

EDIT

So, since my problem was expecting certain trace to appear while the Python application is running, and since I couldn't find a simple way to do so from the application itself, I decided to start the application (as suggested in comments) from another script.

The module I found very helpful, and easier to use than subprocess module, is pexpect module.

... this sounds like the XY problem. You want a piece of your program (a thread?) to wait for a certain "state" of an other piece of your program (an other thread)... the proper way to do that is using synchronization primitives. — Giacomo Alzetta
– Giacomo Alzetta, Commented Aug 27, 2018 at 9:42
Wrong design - specially considering that the logger might be reconfigured to write elsewhere. You'd be better using a proper way of signaling a given condition is satisfied. — bruno desthuilliers
– bruno desthuilliers, Commented Aug 27, 2018 at 9:43
Your script knows when it is logging something, why does it need to read its own logs? — Stop harming Monica
– Stop harming Monica, Commented Aug 27, 2018 at 9:44
It is for the testing purposes. Wait for specific trace, if there is not such trace for about 2 seconds, test fails. — NutCracker
– NutCracker, Commented Aug 27, 2018 at 9:45

vladmihaisima · Accepted Answer · 2018-08-27 10:33:57Z

If you want to do some pre-processing of the logger messages you can do something like:

#!/usr/bin/python

import sys
import logging
import time
import types

def debug_wrapper(self,msg):
  if( hasattr(self,'last_time_seen') and 'message' in msg):
    print("INFO: seconds past since last time seen "+str(time.time()-self.last_time_seen))
  self.last_time_seen = time.time()
  self.debug_original(msg)

logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logger = logging.getLogger("test")

logger.debug_original = logger.debug
logger.debug = types.MethodType(debug_wrapper, logger)

while True:
  logger.debug("INFO: some message.")
  time.sleep(1)

This works by replacing the original debug function of the logger object with your custom debug_wrapper function, in which you can do whatever processing you want, like for example, storing the last time you have seen a message.

Prashant Gupta · Accepted Answer · 2018-09-23 21:26:46Z

1

You can store the script output to a file in real-time and then read its content within the script in real-time(as the contents in the output file is updating dynamically).

To store the script output to a file in real-time, you may use unbuffer which comes with the expect package.

sudo apt-get install expect

Then, while running the script use:

unbuffer python script.py > output.txt

You have to just print the output in the script , which will be dynamically updating to the output file. And hence, read that file each time.

Also, use > for overwriting old or creating new file and >> for appending the contents in previously created output.txt file.

edited Sep 23, 2018 at 21:26

answered Aug 27, 2018 at 9:50

Prashant Gupta

4392 silver badges10 bronze badges

Comments

rocksportrocker · Accepted Answer · 2018-08-27 10:30:19Z

0

If you want to record the output from print statement in other Python code, you can redirect sys.stdout to string like file object as follows:

import io
import sys

def foo():
    print("hello world, what else ?")

stream = io.StringIO()
sys.stdout = stream
try:
    foo()
finally:
    sys.stdout = sys.__stdout__

print(stream.getvalue())

answered Aug 27, 2018 at 10:30

rocksportrocker

7,4792 gold badges34 silver badges51 bronze badges

Collectives™ on Stack Overflow

How to read standard output of a Python script from within it?

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related