How to read twitter-json file and save it as a csv with Python

Question

I have a json-text file containing tweets from a certain hashtag. Now I transform it to a matrix with a row for each tweet and an array of columns such as user, time, latitude, longitude and so on. I have written the following code, but when I get the output file the information is not saved. It has just showed the header row:

#import module
import json
from csv import writer

#input file
tweets = ()
for line in open('file.txt'):
    try: 
        tweets.append(json.loads(line))
    except:
        pass

#variables
ids = [tweet['id_str'] for tweet in tweets]
times = [tweet['created_at'] for tweet in tweets]
users = [tweet['user']['name'] for tweet in tweets]
texts = [tweet['text'] for tweet in tweets]
lats = [(T['geo']['coordinates'][0] if T['geo'] else None) for T in tweets]
lons = [(T['geo']['coordinates'][1] if T['geo'] else None) for T in tweets]
place_names = [(T['place']['full_name'] if T['place'] else None) for T in tweets]
place_types = [(T['place']['place_type'] if T['place'] else None) for T in tweets]

#output file
out = open('tweets_file.csv', 'w')
print >> out, 'id,created,text,user,lat,lon,place name,place type'
rows = zip(ids, times, texts, users, lats, lons, place_names, place_types)
csv = writer(out)
for row in rows:
    values = [(value.encode('utf8') if hasattr(value, 'encode') else value) for value in row]
    csv.writerow(values)
out.close()

Please could you help me to find and clear the bug... Thanks in advance.

R.

SO no es un servicio de revisión de código. Tu programa falla en varios sitios con errores en tiempo de ejecución. Es mejor mostrar respeto por el tiempo de la gente que te puede ayudar de manera gratuita y desinteresada. Antes de postear una pregunta debes isolar el problema y preguntar por lo que falla: no es correcto vomitar todo el codigo y todos los datos y decir "no funcioina". Te envío un enlace que habla sobre este tema: stackoverflow.com/help/mcve Suerte con tu estudio de twitter. — dani herrera
– dani herrera, Commented Nov 12, 2015 at 10:58
¡Muchas gracias! Tranquilo tienes toda la razón en que tengo que ser mucho más específico. Y aunque llevo un tiempecillo por aquí, no soy programador, aunque intento aprender a serlo. — ramiroaznar
– ramiroaznar, Commented Nov 12, 2015 at 11:04

dani herrera · Accepted Answer · 2015-11-12 11:09:55Z

1

In your code, tweets is a tuple:

'tuple' object has no attribute 'append'

It seems you have copy-paste code from several sources without understand what is doing.

import json
from csv import writer

with open('file.txt') as data_file:    
    data = json.load(data_file)

tweets = data['statuses']

ids = [tweet['id_str'] for tweet in tweets]
times = [tweet['created_at'] for tweet in tweets]
...

edited Nov 12, 2015 at 11:09

answered Nov 12, 2015 at 10:33

dani herrera

52.1k11 gold badges133 silver badges188 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

ramiroaznar Over a year ago

Yes, you are right. At the beginning tweets was a list, then I changed it in my effort to find the bug. But as you can observe I did not.

ramiroaznar Over a year ago

¡¡¡Genial!!! Mil gracias. Así que el problema residía en cómo abrir el archivo y no en la declaración de variables...

Collectives™ on Stack Overflow

How to read twitter-json file and save it as a csv with Python

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related