why does this python script not work?

Question

my python script is the following code:

 1 import subprocess
 2 
 3 # initial 'output' to make
 4 r0 = 21
 5 # number of runs to make
 6 rn = 10
 7 
 8 rf = r0+rn
 9 
10 for i in range(r0, rf):
11   #output directory
12   opt_dir = 'output'+str(i)
13   #put it in this output directory
14   popt_dir = './output'+str(i)
15 
16   subprocess.call(['mkdir', opt_dir])
17   subprocess.call(['./exp_fit', 'efit.inp'])
18   subprocess.call(['mv', 'at*', popt_dir])

the intention is this:

i have a program called "exp_fit" which takes an input file "efit.inp". one call to ./exp_fit efit.inp will create output files called 'at0_l0_l0', 'at0_l1_l-1', ... etc (total 475 files starting with 'at').

now, i have been generating data files by running 'exp_fit', then creating output directories and moving them into the output directories with the following bash commands: (for example, with the 20th run of my code)

mkdir output20

mv at* ./output20

so i would think that my script should do the same thing. however, it only does the following:

(1) it correctly generates all output files (475 files starting with 'at') (2) it correctly creates the desired directories (output21 - output30) (3) it DOES NOT, however, correctly move all the output files starting with 'at' into the desired directories. why is this? shouldn't the call to line 18 correctly execute the command to move all my files starting with 'at' into the desired directory?

should i be writing this script with bash instead of python? what is wrong with this?

Eli Bendersky · Accepted Answer · 2012-02-28 13:56:21Z

5

Don't issue subprocess calls for things you can do natively from Python. To move files/dirs around, just use os.rename.

To create a directory, use os.mkdir.

To execute an external program, using subprocess is the right tool.

answered Feb 28, 2012 at 13:56

Eli Bendersky

276k92 gold badges372 silver badges427 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Eli Bendersky Over a year ago

@Kimvais: yes, shutil.move is more general

Community · Accepted Answer · 2017-05-23 10:34:50Z

The problem is that this subprocess command

subprocess.call(['mv', 'at*', './output20'])

is not the same as typing this at a prompt

$ mv at* ./output20

In the latter case, the bash glob expansion converts the single at* argument to a list of arguments of matching filenames for the mv command. So the kernel sees the second as

['mv', 'at0_l0_l0', 'at0_l1_l-1', './output20']

kev's answer tells Python to pass the command through the shell, so the escaping will occur.

But the better solution is to use the glob module and os.rename libraries and not call the subprocess. Creating subprocesses is expensive, and using shell=True could lead to security holes, so it's best to avoid that habit.

(Actually, I suggest making the output directory, switching into it, and then running the exp_fit program from within that directory. Then you won't have to move the output. Try that first.)

kev · Accepted Answer · 2012-02-28 13:56:09Z

1

If shell=True, the executable argument specifies which shell to use.
On Unix, the default shell is /bin/sh.

subprocess.call(['mv', 'at*', popt_dir], shell=True)

answered Feb 28, 2012 at 13:56

kev

163k49 gold badges286 silver badges282 bronze badges

Collectives™ on Stack Overflow

why does this python script not work?

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related