How can I pipe a Python process' output in Python?

Question

I'm writing a program that downloads videos from YouTube, using youtube-dl.

I used to call youtube-dl with subprocess:

import subprocess

p = subprocess.Popen([command], \
    stdout=subprocess.PIPE, \
    stderr=subprocess.STDOUT, \
    universal_newlines = True)

Then, I would read the process' output by calling:

for line in iter(p.stdout.readline, ""):
    hide_some_stuff_using_regex()
    show_some_stuff_using_regex()

However, I prefer using youtube-dl as a Python class. So I'm now doing this:

from youtube_dl import YoutubeDL as youtube_dl

options = {"restrictfilenames": True, \
           "progress_with_newline": True}

ydl = youtube_dl(options)
ydl.download([url])

The code works, but I'm having a hard time finding out, how to pipe youtube-dl's output. Note that I want to use parts of youtube-dl's output to print in real-time, so redirecting sys.stdout to a custom output stream will not work, as I still need sys.stdout to print.

Can you help me?

What is youtube_dl?

user1907906
– user1907906

2015-02-03 11:14:15 +00:00
Commented Feb 3, 2015 at 11:14 — user1907906
– user1907906, Commented Feb 3, 2015 at 11:14
Good catch, updated.

Exeleration-G
– Exeleration-G

2015-02-03 11:36:07 +00:00
Commented Feb 3, 2015 at 11:36 — Exeleration-G
– Exeleration-G, Commented Feb 3, 2015 at 11:36

anon · Accepted Answer · 2015-02-03 12:25:49Z

4

Specifically for youtube-dl, you can set a logger object, like in the advanced example in the documentation:

from youtube_dl import YoutubeDL


class MyLogger(object):
    def debug(self, msg):
        print('debug information: %r' % msg)

    def warning(self, msg):
        print('warning: %r' % msg)

    def error(self, msg):
        print('error: %r' % msg)


options = {
    "restrictfilenames": True,
    "progress_with_newline": True,
    "logger": MyLogger(),
}

url = 'http://www.youtube.com/watch?v=BaW_jenozKc'
with YoutubeDL(options) as ydl:
    ydl.download([url])

answered Feb 3, 2015 at 12:25

anon

Sign up to request clarification or add additional context in comments.

2 Comments

Exeleration-G Over a year ago

Beautiful. Great to be able to be in contact with the developer himself!

LukeSavefrogs Over a year ago

What about getting ffmpeg info? I'd like to grab the the frame= 1766 fps=157 q=-1.0 Lsize= 10152kB time=00:01:13.66 bitrate=1128.9kbits/s speed=6.54x line

Community · Accepted Answer · 2017-05-23 11:53:28Z

1

You can try to redirect sys.stdout to your own output stream
see: https://stackoverflow.com/a/1218951/2134702

To quote the linked answer:

from cStringIO import StringIO
import sys

old_stdout = sys.stdout
sys.stdout = mystdout = StringIO()

# blah blah lots of code ...

sys.stdout = old_stdout

# examine mystdout.getvalue()

And if you want to output to stdout during the redirection, instead of print use old_stdout.write()

edited May 23, 2017 at 11:53

CommunityBot

11 silver badge

answered Feb 3, 2015 at 11:16

Eran

2,4543 gold badges23 silver badges29 bronze badges

4 Comments

user1907906 Over a year ago

This should be a comment, not an answer.

Exeleration-G Over a year ago

I thought about that, but from the moment I redirect sys.stdout to an output stream, I'm not able to use print anymore, am I? The print statements would be redirected to the output stream and not get printed on my screen. That's a problem, because I want to print some parts of youtube-dl's output on my screen in real-time.

Eran Over a year ago

@Exeleration-G, if you save the stdout stream (as shown in the linked answer), you can write to it (instead of using print command), and it would be printed to the main screen in real time.

Exeleration-G Over a year ago

@Eran: Of course! I didn't think of using stuff like old_stdout.write()!

Collectives™ on Stack Overflow

How can I pipe a Python process' output in Python?

2 Answers 2

2 Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related