How to have 2 different function running parallel that return an array in Python

Question

So I have to make a program that searches a keyword through files contained in folder and prints number of times it's used in one. To reduce the search time I've decided to break the folder in 2 folders and have two separate functions that run in parallel, each searches through different folder.

Here is one of the two functions that searches through the files (the other is the same):

    pathText1 = '/Papers/scripts1'
    countmatch1 = 0;
    matrix1 = [[]]
    for filename1 in os.listdir(pathText1):
        fileDir1 = pathText1 + '/' + filename1
        fileText1 = open(fileDir1, "r",encoding='utf8')
        content1 = fileText.read()
        content1 = content1.lower()
        countn1 = content1.count(keyword)
        if count1 > 5:
            print ('The word:   ' + keyword + '    | was found ' + str(count1) + ' times in the file: ' + filename1)
            countmatch1 = countmatch1 + 1
            matrix1.append([filename1,count1])

    print('found ' + countmatch1 + ' matches')
    del matrix1[0]
    return matrix1

So, now I have a problem on how to implement the multiprocessing and make it have the two function return the matrix to the main. Thanks in advance for your help!

Berkay · Accepted Answer · 2019-11-13 11:44:35Z

1

You should use shared variable. Check the example:

from multiprocessing import Process, Manager

def your_function(first_param, second_param):
    # Your work here to search keywords

if __name__ == '__main__':
    manager = Manager()

    # shared variable
    matrices = manager.list()

    jobs = []

    # Range defines how many times you want to call the method.
    for each_process in range(3):
        process = Process(target=your_function, args=(first_param, second_param))
        jobs.append(process)
        process.start()

    for each_process in jobs:
        each_process.join()

    print(matrices)

Also, you can check a more detailed tutorial from here.

answered Nov 13, 2019 at 11:44

Berkay

1,0781 gold badge14 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

foo · Accepted Answer · 2019-11-13 11:35:56Z

0

Have you gone through the multiprocessing library in python3 ? Or are you looking for implementation doubts ?

answered Nov 13, 2019 at 11:35

foo

712 silver badges8 bronze badges

Collectives™ on Stack Overflow

How to have 2 different function running parallel that return an array in Python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related