Multithreaded file read python

Question

import threading


def read_file():
  f = open('text.txt')
  for line in f:
      print line.strip() ,' : ',  threading.current_thread().getName()

if __name__ == '__main__':
  threads = []
  for i in range(15):
    t = threading.Thread(target=read_file)
    threads.append(t)
    t.start()

Question: Will each thread read each line only once from the file above or there are chances that a given thread can end up reading a line twice?

My understanding was that a thread started later will overwrite the file handle for the thread started earlier causing the earlier thread to end up reading few lines twice or thrice or more times.

When I ran this code the outcome was different from what I expected to happen.

Any explanations are welcome.

What was the outcome when you ran the program? Did you notice any demons flying out of your nose? :) — Jeremy Friesner
– Jeremy Friesner, Commented Oct 8, 2016 at 22:41

Martijn Pieters · Accepted Answer · 2016-10-09 12:40:51Z

9

+50

Each thread runs your function independently; each copy of the function opens the file as a local, which is not shared. Each Python file object tracks reading state completely independently; each has their own OS-level file handle here.

So no, if nothing else is altering the file contents, each thread will see each line just once, just the same as if separate processes tried to read the file.

edited Oct 9, 2016 at 12:40

answered Oct 9, 2016 at 12:34

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ankur Agarwal Over a year ago

But why does each thread run the function independently ? It's the same function object being passed to each thread. So how can each thread open the file as a local ?

Martijn Pieters Over a year ago

@abc: yes, you are passing in a function object. So a thread is started and on that thread the function is actually called. When a function is called it creates a new frame with their own locals structure. Each thread calls the function separately, each thread gets their own function frame and each function frame has locals. Those locals are entirely independent.

Collectives™ on Stack Overflow

Multithreaded file read python

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related