3

I need to iterate over the lines of a couple of text files a couple of times. This is currently done with multiple

with open("file.txt") as f: 
    for line in f:
        # do something

Although performance is not an issue yet, I'd like to read the files only once into an io.StringIO buffer and then work with that.

Python io docs:

This is a working snippet

import io
sio = io.StringIO( open("file.txt").read() )
for line in sio:
    print(line)
sio.seek(0)
for line in sio:
    print(line)
sio.close()

or wrapping it in a with statement context manager

import io
with io.StringIO( open("file.txt").read() ) as sio:
    for line in sio:
        print(line)
    sio.seek(0)
    for line in sio:
        print(line)
    #sio.close()

Questions

  1. Is this a "good" way of doing it, what are alternatives?
  2. What happens to the file object used to read the file (there's no way to explicitly close() it this way)?
  3. Where can I read more about Python's io buffering (I think I read something about Python optimizing multiple file accesses by buffering automatically)?

1 Answer 1

1

What you're doing is already the right way. Quoting from this answer: How to read large file, line by line in python

The with statement handles opening and closing the file, including if an exception is raised in the inner block. The for line in f treats the file object f as an iterable, which automatically uses buffered IO and memory management so you don't have to worry about large files.

Sign up to request clarification or add additional context in comments.

1 Comment

Yes, but my questions are about processing a file multiple times. Should it be open()ed multiple times? Also there are no references about the buffering.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.