How to run several scripts in differents folders on Python

Question

I have to run a lot of scripts for paper, and I would like to make this automatic. I have several folders (P1, P2,..., PN) in which I have a script ( test1, test2, ... testN) and I need to run all this, but doing on by one by myself I waste a lot of time that I don't have!

I tried subprocess:

where P1_T1 is:

for i in range(5):
    x = i+2*i
   
    print(x)

and P1_T2 is:

for i in range(5):
    x = i+3*i
    print(x)

But it didn't work.

you need them to run simultaneously is what you mean by it didn’t work? — gold_cy
– gold_cy, Commented Sep 15, 2019 at 15:39
What do you mean by "didn't work"? Did you get an error? If so, what was it? Thanks for clarifying. — ggorlen
– ggorlen, Commented Sep 15, 2019 at 15:40
Also, instead of writing each Popen by hand, you should use a loop instead. — Arnav Borborah
– Arnav Borborah, Commented Sep 15, 2019 at 15:41
@aws_apprentice I don't need to run simultaneously, I just need to run one by one, but this way is not showing any result on the screen, this was just a test, in my real code I have a .txt file as output, and has not been produced with this subprocess code — JCV
– JCV, Commented Sep 16, 2019 at 14:29

Enthus3d · Accepted Answer · 2019-09-16 15:14:15Z

1

If you want to recurse through a set of directories, I recommend using os.walk. This implementation should attempt to run POpen 'python [filename]' on every file in your root directory:

import os
import importlib.util


path = "C:\\SO\\testfolder" # <--- replace this with the path to the folder containing all of your p1, p2, p3, p4 folders.

for root, subdirs, files in os.walk(path):
    for file in files:
        file_path = os.path.join(root, file)
        filename, file_extension = os.path.splitext(file_path)
        if file_extension == ".py":
            print("Now Executing: " + filename + "-----------")
            spec = importlib.util.spec_from_file_location(file, file_path)
            module = importlib.util.module_from_spec(spec)
            spec.loader.exec_module(module)

Edit: Added in use of import library+exec_module to run the python files. Import method referenced from here.

edited Sep 16, 2019 at 15:14

answered Sep 15, 2019 at 15:56

Enthus3d

2,2131 gold badge16 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

JCV Over a year ago

Is there another way to run a script from a folder without POpen?

JCV Over a year ago

This code didn't work, I got the error: raise TypeError("bufsize must be an integer") TypeError: bufsize must be an integer

Enthus3d Over a year ago

I am looking into using importlib to run it, give me a moment to test it

JCV Over a year ago

I figure out what is, it was missing [ ]; This way run without erros but doesnt shows the results : import subprocess import os path = 'C:/Users/Familia/OneDrive - Università di Cagliari/PhD/simpegmaster/Notebooks/Testes_Jenny/TEMFAST/TEMFAST Work/Castanheira/Teste' for root, subdirs, files in os.walk(path): for file in files: subprocess.run(['python',os.path.join(root,file)])

JCV Over a year ago

Let us continue this discussion in chat.

|

Kryštof Řeháček · Accepted Answer · 2019-09-15 15:40:16Z

0

Popen expects list of arguments, so what about

import subprocess
from subprocess import Popen

Popen(['python', 'p1/P1_T1.py'])
time.sleep(1)
Popen(['python','p2/P2_T1.py'])

Does this work for you? There is nice tutorial - Python 3 Subprocess Examples

answered Sep 15, 2019 at 15:40

Kryštof Řeháček

2,5531 gold badge19 silver badges29 bronze badges

1 Comment

JCV Over a year ago

No, this doesn't work. No error, but no result, doesn't show anything on the screen. Like in the photo that I have add

Collectives™ on Stack Overflow

How to run several scripts in differents folders on Python

2 Answers 2

9 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

9 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related