Writing to a CSV file in Python

Question

I'm totally new to Python, but am pretty well-versed in other languages so I have at least some idea of what's going on. I did not write this code, but I am trying to make it work for my purposes.

I'm using an API to retrieve statistics about every play in the NFL from last season. I am trying to write this information to a CSV file, and so far, it's writing the headers to the file and not the actual data.

Can someone tell me what I'm doing wrong?

import csv 
import nflgame 

games = nflgame.games_gen(2013, kind='REG') 
plays = nflgame.combine_plays(games) 
headers = [] 
for statId, info in nflgame.statmap.idmap.iteritems(): 
    headers += info['fields'] 
    headers.sort() 
    headers = ['gsis_id', 'drive_id', 'play_id', 'desc'] + headers 
    writer = csv.DictWriter(open('2013_regular_plays.csv', 'w+'), fieldnames=headers) 
    writer.writerow({k: k for k in headers}) 
    for p in plays: 
        row = {k: getattr(p, k, 0) for k in headers} 
        row['gsis_id'] = p.drive.game.eid 
        row['drive_id'] = p.drive_num 
        row['play_id'] = p.playid 
        row['desc'] = p.desc 
        writer.writerow(row)

@alfasin: note the definition of the writer: it knows how to interpret field names. — Eric O. Lebigot
– Eric O. Lebigot, Commented Sep 7, 2014 at 2:02

Eric O. Lebigot · Accepted Answer · 2014-09-07 02:50:33Z

1

This looks like it should mostly work.

The only detail which is wrong compared to the documentation is that the file should be open in binary mode (w+b).

Also, it is important that the file be closed before you can have a look at it:

with open('2013_regular_plays.csv', 'w+b') as csv_file:
    writer = csv.DictWriter(csv_file, fieldnames=headers)
    …

will automatically close the file after the with block (and the file mode could be more simply wb, if the file is not read inside this block). If you look at the file before it is closed, its contents might still reside in RAM instead of on the disk.

PS: As DSM pointed out, each iteration for statId, info in… empties the CSV file when you open it with the w+ (or w+b) mode. If the last iteration has no plays, then the file end up being empty (with headers only). You typically want to open the CSV file before the loop (maybe even simply with the wb mode).

edited Sep 7, 2014 at 2:50

answered Sep 7, 2014 at 2:03

Eric O. Lebigot

95.1k49 gold badges224 silver badges263 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Nir Alfasi Over a year ago

+1 it's worth mentioning that b mode is necessary only in python 2.

Eric O. Lebigot Over a year ago

@DSM: Yep, it will. w+ implies that the file will also be opened for reading too (as opposed to w, which opens it for writing only: reading causes "IOError: File not open for reading").

Eric O. Lebigot Over a year ago

@alfasin: Yes, with newline='' having to be added to open() in Python 3 (docs.python.org/3.3/library/csv.html#csv.reader).

DSM Over a year ago

.. but this is happening inside a loop. Say, for example, that plays isn't a list but something which can be exhausted on the first pass. Then the for p in plays: loop won't fire afterward, and all that the OP would see at the end would be the header written on the last loop, because using w clobbered everything before.

Collectives™ on Stack Overflow

Writing to a CSV file in Python

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related