Running shell commands sequentially in one shell with subprocess

Question

I am having difficulties running a series of sequential commands using the subprocess module, i need to do this so a python program can call in an installation of a cv virtualenv and then run another python program (that needs to be run within the virtualenv)

This is the command string i run from terminal, you can see it contains multiple commands that run in sequence until the creation of the cv virtual env:

sudo pip install virtualenv virtualenvwrapper && sudo rm -rf ~/.cache/pip && export WORKON_HOME=$HOME/.virtualenvs && source /usr/local/bin/virtualenvwrapper.sh && source ~/.bashrc && mkvirtualenv cv

Running this in the terminal returns me something like this:

(cv) name@computer:~$

from that i can run my python scripts that need the openCV

my code so far is this:

from subprocess import Popen, PIPE, STDOUT

cmd1 = 'sudo pip install virtualenv virtualenvwrapper'
cmd2 = 'sudo rm -rf ~/.cache/pip'
cmd3 = 'export WORKON_HOME=$HOME/.virtualenvs'
cmd4 = 'source /usr/local/bin/virtualenvwrapper.sh'
cmd5 = 'source ~/.bashrc'
cmd6 = 'mkvirtualenv cv'
cmd7 = 'cd /script path'
cmd8 = 'python novo.py'


final = Popen("{}; {}; {}; {}; {}; {}; {}; {}".format(cmd1, cmd2,cmd3,    cmd4, cmd5, cmd6, cmd7, cmd8), shell=True, stdin=PIPE, 
      stdout=PIPE, stderr=STDOUT, close_fds=True)

stdout, nothing = final.communicate()
log = open('log', 'w')
log.write(stdout)
log.close()

And the errors in log look like this:

/bin/sh: 1: source: not found
/bin/sh: 1: source: not found
/bin/sh: 1: mkvirtualenv: not found

How can i achieve a terminal like execution ? again, sequence is crucial.

Community · Accepted Answer · 2017-05-23 12:07:57Z

1

/bin/sh: 1: source: not found

shell=True uses /bin/sh by default. source shell builtin hints at bash. Pass executable='/bin/bash'.

btw, you could use a multiline string literal:

#!/usr/bin/env python3
import sys
from subprocess import check_call, DEVNULL, STDOUT

with open('log', 'wb', 0) as file:
   check_call("""set -e -x
{python} -mpip install --user virtualenv virtualenvwrapper
rm -rf ~/.cache/pip
export WORKON_HOME=$HOME/.virtualenvs
source /path/to/virtualenvwrapper.sh
source ~/.bashrc
mkvirtualenv cv
cd /script path
{python} novo.py
""".format(python=sys.executable),
              shell=True, executable='/bin/bash',
              stdin=DEVNULL, stdout=file, stderr=STDOUT, close_fds=True)

Or save the command into a separate bash script and run the script instead.

DEVNULL is Python 3 feature—it is easy to emulate it on Python 2 too: DEVNULL = open(os.devnull, 'r+b', 0).

edited May 23, 2017 at 12:07

CommunityBot

11 silver badge

answered May 26, 2016 at 12:13

jfs

417k210 gold badges1k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Douglas Heydt Over a year ago

Hello J.F. Sebastian, thank you for your answer, i changed the code on my python script as you suggested, but now when i execute it gives an error "File "programa.py", line 3, in <module> from subprocess import check_call, DEVNULL, STDOUT ImportError: cannot import name DEVNULL" i am really new to this kind of operations, my knowledge stays on python and opencv, dont know how to properly work with this kind of calls and bash files.

Douglas Heydt Over a year ago

i dont really understand how shebangs work but i changed it to pyhton2, it returns a 'CalledProcessError', you can check out the code and error on this image if it not too much trouble postimg.org/image/f6dfdaucr

jfs Over a year ago

My answer fixes one issue: source: not found. If you have another issue; you should ask it as a separate question.

Collectives™ on Stack Overflow

Running shell commands sequentially in one shell with subprocess

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related