Using subprocess for compiling a c++ program from python script

Question

I know there are some similar questions, here Invoking C compiler using Python subprocess command and subprocess, invoke C-program from within Python but I believe my question is in some sense different.

I need to compile a c++ program which uses some ROOT libraries so I need to add some flags and link some libraries for the compilation. Therefore my compilation line on the normal shell is:

> $($ROOTSYS/bin/root-config --cxx) $($ROOTSYS/bin/root-config --cflags --glibs) Analysis.cxx -o analysis.exe

which works nicely. I want to do this compilation from my python script. I have read the documentation for the subprocess module but I could not get a solution without using shell=True in the call of subprocess.Popen and I do not really undestand the difference. If I use:

process = Popen(["$($ROOTSYS/bin/root-config --cxx) $($ROOTSYS/bin/root-config --cflags --glibs) Analysis.cxx -o analysis.exe"], shell=True)

does the job. However, this:

process = Popen(["$($ROOTSYS/bin/root-config --cxx)", "$($ROOTSYS/bin/root-config --cflags --glibs)", "Analysis.cxx", "-o", "analysis.exe"])

I got the following:

    Traceback (most recent call last):
  File "make_posanalysis.py", line 45, in <module>
    "Analysis.cxx", "-o", "analysis.exe"])
  File "Python/2.7.15/x86_64-slc6-gcc62-opt/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "Python/2.7.15/x86_64-slc6-gcc62-opt/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

I would like to understand the difference between using/not using shell=True since it seems to be the reason behind making the script work or not. Or, is there something else I am missing?

Community · Accepted Answer · 2020-06-20 09:12:55Z

2

From the documentation:

If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself. That is to say, Popen does the equivalent of:
Popen(['/bin/sh', '-c', args[0], args[1], ...])

So it'e executing something equivalent to:

/bin/sh -c '$($ROOTSYS/bin/root-config --cxx)' '$($ROOTSYS/bin/root-config --cflags --glibs)' "Analysis.cxx", "-o", "analysis.exe"

This isn't what you want, because it only performs $(...) expansion in the first argument; everything else is taken literally, and become the positional arguments if the command in the first argument refers to $1, $2, etc.

If you want everything parsed by the shell, just give a single string.

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Feb 22, 2019 at 18:46

Barmar

789k57 gold badges555 silver badges669 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Arturo Rodriguez Over a year ago

So if I got it right, there is no way to do it without the shell=True because I need to perform two $(...) expansions?

Barmar Over a year ago

Yes, that's correct. You could execute those commands yourself, save the output, and then build the command from that.

Arturo Rodriguez Over a year ago

So basically the only truly shell interaction (call the shell command, not sure if I can called it "interaction") happens with the first argument if I pass the command as a list. Thanks a lot!

Barmar Over a year ago

If you pass the command as a list, the first element is treated as the shell command, the rest are arguments. If you pass it as a string, the whole string is a shell command.

Collectives™ on Stack Overflow

Using subprocess for compiling a c++ program from python script

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related