0

I'm reading a tutorial about python for very beginners and at some point the author define some ways to work with files. My doubt is related to memory management and file arrays.

#open a file for reading
file = open(filename, 'r')
#this turns the file into an array. 
lines = file.readlines() `

Python is smart enough to check the file size? What happens if the file has about 1 GB of data? Python will throw the entire file to the memory (array)? Or this is a lazy operation, just like C/C++ does!

Thanks in advance.

Improve this question
2
  • 1
    Does the tutorial use file as a variable name? Commented Mar 21, 2014 at 6:36
  • 1
    Yes. I know this isn't a good way to define a variable name because file is a type-object in python. Commented Mar 21, 2014 at 6:39

2 Answers 2

Reset to default
4

Quoting the Python Tutorial:

To read a file’s contents, call f.read(size), which reads some quantity of data and returns it as a string. size is an optional numeric argument. When size is omitted or negative, the entire contents of the file will be read and returned; it’s your problem if the file is twice as large as your machine’s memory. Otherwise, at most size bytes are read and returned. If the end of the file has been reached, f.read() will return an empty string ("").

It also applies for readlines(), so Python will "throw" the entire file to the memory.

Also, by quoting Python Docs:

file.readlines([sizehint]): Read until EOF using readline() and return a list containing the lines thus read. ...

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

Python is smart enough to check the file size?

Yes. There are some OS related function. You can get file size using them.

What happens if the file has about 1 GB of data? Python will throw the entire file to the memory (array)? Or this is a lazy operation, just like C/C++ does!

If you use fp.readlines, yes 1GB of data will be stored in memory. But unsurprisingly there is a such function which increase file pointer like C/C++. So you can read data chunk by chunk and reduce memory usage.

fp.read(n)

I think this is the C/C++ like file operation you are saying.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.